Information processing apparatus and information processing method for controlling gesture operations based on postures of user

ABSTRACT

An information processing apparatus according to an embodiment of the present technology includes a physical information acquisition unit, a gesture information acquisition unit, a determination condition setting unit, and a control unit. The physical information acquisition unit acquires physical information of a user wearing a head-mounted casing. The gesture information acquisition unit acquires gesture information regarding a gesture of the user. The determination condition setting unit sets, on the basis of the physical information, a determination condition for determining a trigger gesture that is a trigger for an operation input by the gesture of the user. The control unit determines, on the basis of the gesture information and the determination condition, whether or not the trigger gesture is performed and controls, on the basis of a result of the determination, an output corresponding to the trigger gesture.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a U.S. National Phase of International PatentApplication No. PCT/JP2020/018497 filed on May 7, 2020, which claimspriority benefit of Japanese Patent Application No. JP 2019-101104 filedin the Japan Patent Office on May 30, 2019. Each of the above-referencedapplications is hereby incorporated herein by reference in its entirety.

TECHNICAL FIELD

The present technology relates to an information processing apparatus,an information processing method, and a program that are capable ofgesture operation.

BACKGROUND ART

Conventionally, there is known a technology that detects a motion ofeach site of a user. For example, an apparatus such as a head-mounteddisplay (HMD) worn by the user detects a user motion and performsdisplay control or the like of an image by using a detection result.

For example, Patent Literature 1 has described an HMD that combines anddisplays electronic information on an outside-world image observed in afield-of-view of a user. This HMD detects the user's head motion tothereby control display/non-display of an image display element thatdisplays the electronic information. For example, in a case where theangular velocity or movement velocity of the user's head is above apredetermined threshold, the image display element is switched to thenon-display. Accordingly, when the user starts walking or the like, theelectronic information displayed in front of the eyes is not displayed,so that the field-of-view of the user can be ensured (paragraphs [0027],[0035], and [0038] in the specification, FIGS. 1 and 5, etc. of PatentLiterature 1).

CITATION LIST Patent Literature

-   Patent Literature 1: Japanese Patent Application Laid-open No. HEI    11-161190

DISCLOSURE OF INVENTION Technical Problem

In recent years, there have been developed technologies that detectmotions of user's head, arms, legs, and the like to thereby enableoperation inputs based on gestures or the like to be performed. Thegesture operations are expected to be applied in various scenes, and itis desirable to provide a technology capable of improving theoperability of gesture operations.

In view of the above-mentioned circumstances, it is an object of thepresent technology to provide an information processing apparatus, aninformation processing method, and a program that can improve theoperability of gesture operations.

Solution to Problem

In order to accomplish the above-mentioned object, an informationprocessing apparatus according to an embodiment of the presenttechnology includes a physical information acquisition unit, a gestureinformation acquisition unit, a determination condition setting unit,and a control unit.

The physical information acquisition unit acquires physical informationof a user wearing a head-mounted casing.

The gesture information acquisition unit acquires gesture informationregarding a gesture of the user.

The determination condition setting unit sets, on the basis of thephysical information, a determination condition for determining atrigger gesture that is a trigger for an operation input by the gestureof the user.

The control unit determines, on the basis of the gesture information andthe determination condition, whether or not the trigger gesture isperformed and controls, on the basis of a result of the determination,an output corresponding to the trigger gesture.

In this information processing apparatus, the determination condition ofthe trigger gesture that is the trigger for the operation input by thegesture is set on the basis of the physical information of the userwearing the head-mounted casing. Whether or not the trigger gesture isperformed is determined on the basis of this determination condition andthe gesture information, and the output corresponding to the triggergesture is controlled on the basis of the determination result.Accordingly, the trigger gesture performed by the user is determined inaccordance with the physical information of the user, and therefore theoperability of gesture operations can be improved.

The determination condition setting unit may set the determinationcondition on the basis of physical feature information of the user, thephysical feature information corresponding to the physical information.

Accordingly, for example, a determination condition appropriate to aphysical feature of the user can be set, and for example, the physicalburden in performing the trigger gesture can be sufficiently reduced.

The physical feature information may include information regarding atleast one of a field-of-view feature or a motion feature of the user.

Accordingly, for example, the determination condition is set inaccordance with a field-of-view range, a motion range, and the like in aphysical state of the user, and the user can easily perform a motionthat is the trigger gesture.

The determination condition may include a determination threshold fordetermining the trigger gesture.

Accordingly, the trigger gesture can be easily distinguished from othergestures.

The physical information acquisition unit may acquire a posture of theuser as the physical information. In this case, the determinationcondition setting unit may set the determination condition in accordancewith the posture of the user.

Accordingly, for example, a determination condition appropriate to theposture of the user is set, and a motion that is the trigger gesture canbe easily performed and the operability can be sufficiently improved.

The determination condition setting unit may set the determinationcondition in accordance with at least one of a field-of-view feature ora motion feature in the posture of the user.

Accordingly, for example, a determination condition appropriate to afield-of-view range and a motion range at each timing is set. As aresult, the physical burden in performing a motion that is the triggergesture can be greatly reduced, and excellent operability can beprovided.

The trigger gesture may be a head gesture of the user moving the headupward. In this case, the determination condition may include athreshold angle with respect to a posture angle of the head of the user,the posture angle changing in a manner that depends on the head gesture.Moreover, the physical information acquisition unit may detect astanding state or a seated state as the posture of the user. Moreover,the determination condition setting unit may set, in a case where theseated state is detected, the threshold angle to be a smaller value thanin a case where the standing state is detected.

For example, in the seated state, the line-of-sight tends to movedownward as compared with the standing state. As described above, bysetting the threshold angle, the burden on the body of the user whoperforms the gesture is reduced. Moreover, malfunctions in which motionsthat the user does not intend are determined as triggers and the likecan be reduced.

The head-mounted casing may be a casing for a head-mounted display thatdisplays a virtual image. In this case, the control unit may control adisplay position of the virtual image on the basis of the physicalinformation.

Accordingly, the display position of the virtual image is adjusted inaccordance with the physical information, and therefore display or thelike easy for the user to view is realized. Accordingly, the operabilitycan be sufficiently improved.

The physical information acquisition unit may acquire a posture of theuser as the physical information. In this case, the control unit maycontrol the display position of the virtual image in accordance with theposture of the user.

Accordingly, for example, the virtual image can be displayed at aposition that the user can easily visually recognize irrespective of theposture of the user.

The virtual image may be an upper image to be displayed on a diagonallyupper side as viewed from the user. In this case, the physicalinformation acquisition unit may detect a standing state or a seatedstate as the posture of the user. Moreover, the control unit may set, ina case where the seated state is detected, the display position of theupper image to be a lower position than in a case where the standingstate is detected.

Accordingly, the upper image can be displayed at a position easy for theuser to view irrespective of the standing state and the seated state,and the physical burden can be sufficiently reduced.

The physical information acquisition unit may acquire a height of theuser as the physical information. In this case, the virtual image may bea lower image to be displayed on a diagonally lower side as viewed fromthe user. Moreover, the control unit may set a display position of thelower image in accordance with the height of the user.

Accordingly, the lower image can be displayed at a proper positionirrespective of the height of the user, and the physical burden can besufficiently reduced.

The information processing apparatus may further include anenvironmental information acquisition unit that acquires environmentalinformation regarding a surrounding environment of the user.

Accordingly, for example, a change or the like in the physical featureof the user due to a change in the surrounding environment can bereflected to each control.

The determination condition setting unit may set the determinationcondition on the basis of the environmental information.

Accordingly, the determination condition can be adjusted in accordancewith the physical feature of the user or the like that depends on thesurrounding environment of the user, and the operability can besufficiently improved.

The trigger gesture may be a head gesture of the user moving the headupward. In this case, the determination condition may include athreshold angle with respect to a posture angle of the head of the user,the posture angle changing in a manner that depends on the head gesture.Moreover, the environmental information acquisition unit may detectpresence/absence of a ceiling in the surrounding environment. Moreover,the determination condition setting unit may set, in a case where theceiling is detected, the threshold angle to be a smaller value than in acase where the ceiling is not detected.

For example, in an environment where the ceiling is present, theline-of-sight tends to move downward as compared with an environmentwhere the ceiling is absent. As described above, by setting thethreshold angle, the burden on the body of the user who performs thegesture is reduced. Moreover, malfunctions in which motions that theuser does not intend are determined as triggers and the like can bereduced.

The head-mounted casing may be a casing for a head-mounted display thatdisplays a virtual image. In this case, the control unit may control adisplay position of the virtual image on the basis of the environmentalinformation.

Accordingly, the display position of the virtual image is adjusted inaccordance with the surrounding environment, and therefore display orthe like easy for the user to view is realized. Accordingly, theoperability can be sufficiently improved.

The environmental information acquisition unit may determine whether ornot the surrounding environment includes a vanishing point. In thiscase, the control unit may set, in a case where the surroundingenvironment includes the vanishing point, the display position of thevirtual image by using the vanishing point as a basis.

Accordingly, for example, the virtual image can be arranged in a rangeon which the lines-of-sight of the user concentrate, and theaccessibility to the virtual image can be improved.

The environmental information acquisition unit may detect a position ofa floor as the environmental information. In this case, the control unitmay set the display position of the virtual image in accordance with theposition of the floor.

Accordingly, for example, the virtual image can be displayed at aposition easy to visually recognize irrespective of the height of thefloor, and the accessibility to the virtual image can be improved.

The head-mounted casing may be a casing for a head-mounted display thatdisplays a virtual image. In this case, the trigger gesture may functionas a trigger for a display operation of causing the virtual image to bedisplayed, a trigger for a non-display operation of causing the virtualimage not to be displayed, or a trigger for a selection operation ofselecting the virtual image.

Accordingly, the operability in causing the virtual image to bedisplayed or not to be displayed, selecting the virtual image, and thelike can be improved.

An information processing method according to an embodiment of thepresent technology is an information processing method to be performedby a computer system and includes acquiring physical information of auser wearing a head-mounted casing.

Gesture information regarding a gesture of the user is acquired.

A determination condition for determining a trigger gesture that is atrigger for an operation input by the gesture of the user is set on thebasis of the physical information.

Whether or not the trigger gesture is performed is determined on thebasis of the gesture information and the determination condition, andoutput corresponding to the trigger gesture is controlled on the basisof a result of the determination.

A program according to an embodiment of the present technology causes acomputer system to execute the following steps.

A step of acquiring physical information of a user wearing ahead-mounted casing.

A step of acquiring gesture information regarding a gesture of the user.

A step of setting, on the basis of the physical information, adetermination condition for determining a trigger gesture that is atrigger for an operation input by the gesture of the user.

A step of determining, on the basis of the gesture information and thedetermination condition, whether or not the trigger gesture is performedand controlling output corresponding to the trigger gesture on the basisof a result of the determination.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 A schematic diagram describing the outline of an HMD according tothe embodiment of the present technology.

FIG. 2 A block diagram showing a configuration example of the HMD shownin FIG. 1 .

FIG. 3 A schematic diagram for describing an example of a virtual image.

FIG. 4 A flowchart showing an example of a basic operation of the HMD.

FIG. 5 A schematic diagram showing an example of a virtual imagedisplayed by an upward gesture.

FIG. 6 A schematic diagram showing an example of features relating to ahuman vertical field-of-view.

FIGS. 7A and 7B Schematic diagrams for describing setting of adetermination threshold appropriate to a posture of a user.

FIGS. 8A and 8B Schematic diagrams showing a display example of an upperimage corresponding to the environmental information.

FIG. 9 A schematic diagram showing an example of features relating to ahuman horizontal field-of-view.

FIG. 10 A schematic diagram showing a display example of a virtual imagecorresponding to the environmental information.

FIG. 11 A schematic diagram showing an example of features relating to ahuman range-of-motion.

FIG. 12 A schematic diagram showing an example of the display positionof the virtual image displayed on a diagonally lower side as viewed fromthe user.

FIGS. 13A and 13B Schematic diagrams showing an example of a lowerimage.

MODE(S) FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments of the present technology will be describedwith reference to the drawings.

[Configuration of HMD]

FIG. 1 is a schematic diagram describing the outline of an HMD accordingto an embodiment of the present technology. A head-mounted display (HMD)100 is an eye-glasses-type apparatus including a see-through display andis used mounted on the head of a user 1. The user 1 wearing the HMD 100is enabled to visually recognize a scene in reality and to visuallyrecognize an image displayed on the see-through display at the sametime. That is, the use of the HMD 100 enables a virtual image to bedisplayed superimposed on a space in reality (real space) surroundingthe user 1. Accordingly, the user 1 can experience augmented reality(AR) or the like. The HMD 100 has a head-mounted casing 5. Thehead-mounted casing 5 is an outer casing of the HMD 100, and isconfigured to be capable of being mounted on a human head. A controllerto be described later and the like (see FIG. 2 ) are mounted on thiscasing 5, so that the HMD 100 is configured. That is, the entireapparatus including the head-mounted casing 5, the controller, and thelike is the HMD 100. The type of the head-mounted casing 5 (HMD 100) isnot limited, and for example, the eye-glasses-type casing 5 may bereplaced by an immersive casing or the like that is arranged to coverthe head of the user 1. Alternatively, a head-mounted casing or the likeconfigured to support a portable display apparatus (smartphone, gameconsole, or the like) in front of the eyes of the user 1 may be used.

The user 1 who uses the HMD 100 can perform various operation inputs(gesture operations) through gestures. Here, a gesture of the user 1means a motion of the user 1. For example, motions (extension and flex)of the user 1 tilting the head upward and downward, motions (lateralflex) of the user 1 tilting the head leftward and rightward, a motion(rotation) of the user 1 rotating the head (neck), and the like areincluded in gestures of the user 1 using the head. Moreover, motions ofthe user 1 moving the arms, legs, and the like, motions of the user 1moving the entire body, e.g., squatting down, seating, and standing, ormotions of the user 1 moving the line-of-sight, motions of the user 1opening and closing the eyelids, a motion of the user 1 blinking, andthe like are also included in the gestures of the user 1. Furthermore,gestures that are combinations of those motions can be performed.

When the user 1 wearing the HMD 100 performs a predetermined gesture,operation processing corresponding to the gesture is performed. Forexample, processing of switching display/non-display of a predeterminedvirtual image or the like is performed corresponding to a gesture of theuser 1 tilting the head upward or downward. Alternatively, processing ofselecting a target from a plurality of virtual images or the like isperformed corresponding to a gesture of the user 1 rotating the headleftward or rightward. In addition, the types and the like of gesturesand operation processing are not limited. As described above, in the HMD100, various types of operation processing can be performed byoperations of the user 1 him or herself, and virtual experience having ahigh degree of freedom can be realized.

FIG. 2 is a block diagram showing a configuration example of the HMD 100shown in FIG. 1 . The HMD 100 includes a display unit 10, a speaker 11,a communication unit 12, an operation input unit 13, a sensor unit 14, astorage unit 15, and a controller 20.

The display unit 10 is a see-through display that displays a virtualimage, and arranged to cover at least a part of a field-of-view of theuser 1. In the HMD 100 shown in FIG. 1, see-through displays for theleft and right eyes are arranged in front of the left and right eyes ofthe user 1, respectively. It should be noted that the present technologyis not limited to the configuration to display a virtual image to botheyes of the user 1, and for example, the display unit 10 may beconfigured to display a virtual image only to one eye of the user 1. Asthe display unit 10, for example, a see-through organic EL display,liquid-crystal display, or the like is used. In addition, the specificconfiguration of the display unit 10 is not limited, and a see-throughdisplay of an arbitrary type such as a type that projects an image on atransparent screen and a type that displays an image by the use of aprism and the like may be used.

The speaker 11 is, for example, arranged near the ear of the user 1, andreproduces audio signals in accordance with, for example, content beingexecuted in the HMD 100. The communication unit 12 is a communicationmodule for sending and receiving data to/from another apparatus with awire or wirelessly. The communication unit 12 performs communicationbetween the HMD 100 and an external apparatus or the like by, forexample, a method such as a wired local area network (LAN), a wirelessLAN, wireless fidelity (Wi-Fi) (registered trademark), infraredcommunication, Bluetooth (registered trademark), andshort-distance/contactless communication. The operation input unit 13has a physical operation mechanism such as a switch, a button and alever. For example, by the user 1 operating the operation input unit 13,various operations are received.

The sensor unit 14 acquires various types of information regarding theuser 1 and the surrounding environment of the user. The sensor unit 14includes a rear-facing camera 16, a 9-axis sensor 17, a position sensor18, and a biometric sensor 19. In FIG. 1 , the rear-facing camera 16 isschematically shown as an example of the sensor unit 14. Moreover, the9-axis sensor 17, the position sensor 18, and the biometric sensor 19are arranged as appropriate, for example, at predetermined positionsinside the casing constituting the HMD 100. It should be noted that thetypes, the number, and the like of the sensors provided in the sensorunit 14 are not limited, and for example, an atmospheric sensor, atemperature sensor, and the like may be provided.

The rear-facing camera 16 is arranged to face the front of the HMD 100and takes an image in the real space that is included in thefield-of-view of the user 1. As the rear-facing camera 16, for example,a digital camera including an image sensor such as a CMOS sensor and aCCD sensor is used. Alternatively, for example, a stereo camera capableof detecting depth data or the like in the real space or a cameraequipped with a time of flight (TOF) sensor or the like may be used asthe rear-facing camera 16. The image data or depth data detected by therear-facing camera 16 is used for various types of recognitionprocessing such as object recognition in the real space and recognitionof hands of the user 1. Moreover, such data is used for processing suchas simultaneous localization and mapping (SLAM) that estimates theself-position of the user 1 and generates a surrounding-space map or thelike. The specific configuration of the rear-facing camera 16 is notlimited, and for example, an arbitrary camera sensor capable of sensingthe real space at a desired accuracy may be used as the rear-facingcamera 16.

The 9-axis sensor 17 includes a 3-axis acceleration sensor, a 3-axisgyro sensor, and a 3-axis compass sensor. The 3-axis acceleration sensordetects acceleration in three axial directions of the HMD 100. The3-axis gyro sensor detects angular velocity (rotation velocity) aroundthe three axes of the HMD 100. The 3-axis compass sensor detectsgeomagnetic force (direction) in the three axial directions. Each typeof data detected by the 9-axis sensor 17 is used for processing ofdetecting a posture, a motion, and the like of the head of the user 1wearing the HMD 100 and processing of recognizing behaviors of the user1, such as a standing motion and a seating motion. Moreover, such datais used for detecting the posture, the orientation, and the like of theuser 1 in the space map configured by the use of the SLAM or the like.

The position sensor 18 detects a current position of the HMD 100 (user1) on the basis of an externally acquired signal. For example, theposition sensor 18 is configured as a global positioning system (GPS)position-measuring unit, and receives radio waves from a GPS satelliteblock and detects a position at which the HMD 100 is present. Moreover,the position sensor 18 may be configured to detect the position by, forexample, Wi-Fi (registered trademark), Bluetooth (registered trademark),sending and receiving to/from a portable phone, a PHS, a smartphone, orthe like, short-distance communication, or the like other than the GPS.Alternatively, for example, the position of the HMD 100 may be detectedby detecting makers or the like arranged in the periphery of the user 1.In this case, the position sensor 18 can be realized using theabove-mentioned rear-facing camera 16 and the like. In addition, thespecific configuration of the position sensor 18 is not limited.

The biometric sensor 19 detects biometric data. For example, as thebiometric sensor 19, a front-facing camera that takes images of the eyesof the user 1 is used. The eye image is used for, for example,processing of detecting a line-of-sight direction, a gazing time, andthe like of the user 1. Alternatively, for example, the biometric sensor19 that detects a body movement, a body position, and the like of theuser 1 may be mounted and a detection result thereof may be used forprocessing of detecting a posture and the like of the user 1. Inaddition, the type and the like of the biometric sensor 19 are notlimited, and for example, a sensor capable of detecting a pulse rate, abody temperature, sweat, a blood pressure, sweat, pulses, respiration,eyeblink, an eye movement, a pupil diameter, a blood pressure, brainwaves, a skin temperature, electric skin resistance, micro vibration(MV), myopotential potential, blood oxygen saturation (SpO₂), or thelike may be used as appropriate.

The storage unit 15 is a nonvolatile storage device, and for example, ahard disk drive (HDD), a solid state drive (SSD), or the like is used.The storage unit 15 stores map data and a control program. The map datais data that functions as map data relating to the real space, and forexample, the space map (three-dimensional model of the surroundingenvironment) or the like generated by the use of the SLAM or the like isused. The control program is a program for controlling operations of theentire HMD 100. The method of installing the map data and the controlprogram into the HMD 100 is not limited.

The controller 20 controls the operation of each block of the HMD 100.The controller 20 has a hardware configuration necessary for a computer,such as a CPU and a memory (RAM, ROM), for example. By the CPU loadingthe control program stored in the storage unit 15 into the RAM andexecuting it, various types of processing are performed. The specificconfiguration of the controller 20 is not limited, and for example, aprogrammable logic device (PLD) such as a field programmable gate array(FPGA) or another device such as an application specific integratedcircuit (ASIC) may be used as the controller 20. The controller 20corresponds to an information processing apparatus according to thisembodiment. Moreover, the HMD 100 is an apparatus in which thecontroller 20 is mounted on the head-mounted casing 5. Thus, in thisembodiment, it can also be said that the HMD 100 itself functions as theinformation processing apparatus.

In this embodiment, by the CPU of the controller 20 executing theprogram (control program) according to this embodiment, a physicalinformation acquisition unit 21, a gesture information acquisition unit22, an environmental information acquisition unit 23, a determinationcondition setting unit 24, a gesture detection unit 25, a layoutcalculation unit 26, a layout determination unit 27, and an outputcontrol unit 28 are realized as functional blocks. Then, an informationprocessing method according to this embodiment is performed by thosefunctional blocks. It should be noted that dedicated hardware such as anintegrated circuit (IC) may be used as appropriate in order to realizethe respective functional blocks.

The physical information acquisition unit 21 acquires physicalinformation of the user 1 wearing the HMD 100. The physical informationis information regarding the body of the user 1. The physicalinformation includes, for example, information that changes over time,such as posture and motion of the user 1, and information specific tothe user 1, such as age and gender of the user 1. Such physicalinformation can represent a physical state of the user 1 wearing the HMD100. That is, it can also be said that the physical information isinformation indicating the physical state of the user 1.

The physical information acquisition unit 21 acquires a posture of theuser 1 as the physical information. Specifically, posture detectionprocessing of detecting the posture of the user 1 is performed on thebasis of an output result of each sensor or the like included in theabove-mentioned sensor unit 14. In this embodiment, the physicalinformation acquisition unit 21 detects a standing state or a seatedstate as the posture of the user 1.

For example, by monitoring acceleration output from the 9-axis sensor17, acceleration or the like at the time when the user 1 stands up orsits down is detected and a current posture (standing or sitting) of theuser 1 is detected. Alternatively, the standing state or the seatedstate may be detected, for example, using changes in the pulse rate andblood pressure measured by the biometric sensor 19, a change in theatmospheric pressure, or the like. Moreover, integrating those sensoroutputs, the posture of the user 1 can also be detected. In addition,processing of determining whether the user 1 is standing or seated maybe performed by machine learning or the like. It should be noted thatother than the standing state and the seated state, a state (laying-downstate) in which the user 1 is laying down, a state in which the user 1is squatting down, and the like may be detected. The method of detectingthe posture of the user 1 is not limited.

Moreover, the physical information acquisition unit 21 acquires a heightof the user 1 as the physical information. Typically, theabove-mentioned storage unit 15 or the like stores the height of theuser 1 in advance, and the physical information acquisition unit 21reads in the height of the user 1 as appropriate. Alternatively, theheight of the user 1 may be estimated by monitoring a height position ofthe HMD 100 from the floor on the basis of a detection result of therear-facing camera 16 or the like. The method of acquiring the height ofthe user 1 is not limited.

In addition, as the physical information, a motion state of the user 1,such as a walking state and a running state, may be acquired. The motionstate of the user 1 can be detected on the basis of the outputs of the9-axis sensor 17, the position sensor 18, and the biometric sensor 19.Alternatively, as the physical information, the age, the gender, and thelike of the user 1 may be acquired. Such information specific to theuser 1 is, for example, pre-stored in the storage unit 15 as userinformation, and the physical information acquisition unit 21 reads init as appropriate. Each piece of physical information acquired by thephysical information acquisition unit 21 is output to the determinationcondition setting unit 24, the layout calculation unit 26, and the liketo be described later.

The gesture information acquisition unit 22 acquires gesture informationregarding a gesture of the user. The gesture information is informationfor detecting the gesture of the user 1. Acceleration, angular velocity,direction, and the like output from the 9-axis sensor 17 are, forexample, gesture information for detecting gestures of the user 1 usingthe head, gestures of the user 1 moving the entire body, and the like.Moreover, images or the like of the hands and legs of the user 1, whichthe rear-facing camera 16 has taken, are gesture information fordetecting gestures using the hands, gestures using the legs, and thelike. Moreover, images of the eyes of the user 1 are gesture informationfor detecting gestures of the eyes of the user 1. In addition, the typesand the like of gesture information are not limited. For example, in acase where an external camera (not shown) that takes images of the user1, for example, is provided separate from the HMD 100, images of theuser 1 that the external camera has taken are gesture information. Eachpiece of gesture information acquired by the gesture informationacquisition unit 22 is output to the gesture detection unit 25 to bedescribed later.

The environmental information acquisition unit 23 acquires environmentalinformation regarding the surrounding environment of the user 1. Theenvironmental information includes, for example, image data of thesurrounding environment, which the rear-facing camera 16 has taken,depth data, and the like. Moreover, the environmental informationacquisition unit 23 performs processing such as the SLAM on the basis ofthe image data, depth data, and the like and generates a space map orthe like of the surrounding environment of the user 1. This space map isenvironmental information indicating a position(s) or the like of a realobject(s) in the surrounding environment of the user 1. Alternatively,for example, in a case where a space map or the like is pre-stored, thespace map may be read in on the basis of the output (positioninformation and the like) of the position sensor 18. Alternatively, onthe basis of the space map or the like, detailed environmentalinformation may be generated. For example, information indicating thatthe current position of the user 1 is indoor or outdoor, heights of theceiling and floor, and the like are detected as the environmentalinformation. It should be noted that the present technology is notlimited to the case where the space map is used, and whether or not theuser 1 is indoor or outdoor may be detected on the basis of the positioninformation of the user 1, the image data of the rear-facing camera, andthe like. In addition, the type and the like of the environmentalinformation are not limited, and for example, brightness, temperature,and the like of the surrounding environment may be acquired as theenvironmental information. Each piece of environmental informationacquired by the environmental information acquisition unit 23 is outputto the determination condition setting unit 24, the layout calculationunit 26, and the like to be described later.

On the basis of the physical information, the determination conditionsetting unit 24 sets a determination condition for determining a triggergesture that is a trigger for an operation input by a gesture of theuser 1. In the operation input by the gesture (gesture operation), theuser 1 performs a predetermined gesture to thereby perform operationprocessing corresponding to the gesture. The operation processing isstarted by detecting the predetermined gesture by the use of the sensorunit 14 or the like. Thus, in the gesture operation, a gesture of theuser 1 him or herself functions as the trigger. The determinationcondition for determining this gesture (trigger gesture) that functionsas the trigger is set on the basis of the physical information such asthe posture and height of the user 1.

Physical Feature=Physical Feature Information

In this embodiment, the determination condition setting unit sets thedetermination condition on the basis of physical feature information ofthe user, which corresponds to the physical information. The physicalfeature information is, for example, information regarding physicalfeature of the user 1 in the physical state indicated by the physicalinformation. For example, a human physical feature (human feature)includes features that change depending on a physical state (posture,height, age, gender, etc.). Therefore, it is conceivable that a rangethat the user 1 can visually recognize, the tendence of the gesture, andthe like change depending on a change in the posture of the user 1 whouses the HMD 100, an individual difference of the user 1, and the like.

The physical feature information includes information regarding afield-of-view feature of the user in the physical state. For example, ina case where the user 1 is seated (seated state), the field-of-view(line-of-sight) tends to move downward as compared with a case where theuser 1 is standing (standing state) (see FIG. 6 ). In this case,information indicating a field-of-view range in each posture is thephysical feature information. Moreover, the physical feature informationincludes information regarding a motion feature of the user 1 in thephysical state. For example, between the seated state and the standingstate, a range in which the head, the body, and the like can move isdifferent (see FIG. 11 ). In this case, information indicating a rangein which motions are allowed in each posture is the physical featureinformation. In addition, in a case where the state such as the height,the age, and the gender is different, the physical feature such as thefield-of-view range and the range in which motions are allowed isdifferent. The determination condition setting unit 24 sets adetermination condition of the trigger gesture in accordance with thephysical feature information in the physical state such as the postureand the height acquired as the physical information.

Moreover, in this embodiment, the determination condition setting unit24 sets the determination condition on the basis of the environmentalinformation. For example, between the indoor and outdoor environments, arange of motions that the user 1 unconsciously performs, theline-of-sight, and the like are different. That is, in a case where theuser 1 performs a certain gesture, the amount of motion (size of motion)in performing the gesture can differ depending on the surroundingenvironment. In addition, it is conceivable that the motion (gesture) ofthe user 1 changes depending on the surrounding environment. Thedetermination condition setting unit 24 sets the determination conditionof the trigger gesture in accordance with human physiological andphysical features that occur in such an external environment factor.

In this embodiment, the determination condition includes a determinationthreshold for determining the trigger gesture. That is, in order todiscriminate the trigger gesture from other gestures, thresholdprocessing using the determination threshold is performed. Thedetermination threshold is, for example, a threshold with respect to aparameter that changes when the trigger gesture is performed. Forexample, in a case where the trigger gesture is a gesture that changesthe posture of the head, a threshold angle with respect to a postureangle of the head (HMD 100) is the determination threshold. Thus, adetermination threshold corresponding to a trigger gesture that is atarget is used for determining the trigger gesture. Alternatively, aplurality of determination thresholds may be used for determining asingle trigger gesture. Setting processing of the determinationthreshold by the determination condition setting unit 24 will bedescribed later in detail.

The gesture detection unit 25 detects a gesture of the user 1 on thebasis of the gesture information acquired by the gesture informationacquisition unit 22. For example, on the basis of the acceleration orthe like of the HMD 100 that the sensor unit 14 has detected, a gesture(head gesture) of the user 1 changing the posture and position of thehead or the like is detected. Moreover, a gesture (hand gesture)performed by the user 1 moving the hands or the like is detected fromimage data taken by the rear-facing camera 16, for example. The methodof detecting a gesture is not limited, and for example, a motionrecognition technology, an image recognition technology, and the likeusing the 9-axis sensor 17 may be used. Alternatively, gesture detectionusing machine learning or the like may be performed.

Moreover, the gesture detection unit 25 determines whether or not theuser 1 has performed the trigger gesture. That is, the gesture detectionunit 25 determines whether or not the gesture indicated by the gestureinformation of the user 1 (motion of the user 1) is the trigger gesture.

For determining the trigger gesture, the determination condition(determination threshold) set by the determination condition settingunit 24 is used. For example, it is assumed that a gesture of moving thehead upward is set as the trigger gesture. In this case, whether or notthe trigger gesture is performed is determined by determining whether ornot the angle of tilt of the head of the user 1 exceeds thecorresponding determination threshold (threshold angle). In this manner,on the basis of the gesture information and the determination condition,the gesture detection unit 25 determines whether or not the triggergesture is performed. The use of the determination condition enablesintended gestures and other motions of the user 1 to be distinguishedfrom each other and detected.

The layout calculation unit 26 acquires data of a virtual image to bedisplayed on the display unit 10 (see-through display) and calculates alayout of the virtual image. For example, in a case where a triggergesture made by the user 1 is detected, data of a virtual imagecorresponding to the trigger gesture is read in and a layout of thevirtual image is calculated. That is, on the basis of the determinationresult of the trigger gesture, the layout calculation unit 26 controlsthe output of the virtual image corresponding to the trigger gesture.Moreover, in accordance with the progress of the content being executedin the HMD 100, the layout calculation unit 26 reads in the data of thevirtual image and calculates a layout thereof.

The data of the virtual image is, for example, stored in the storageunit 15 as content information. Alternatively, the content informationmay be read in via the communication unit 12 or the layout calculationunit 26 may generate and edit the data of the virtual image. It shouldbe noted that the number and the like of virtual images to be displayedon the display unit 10 are not limited. For example, there are a casewhere a single virtual image is displayed, a case where a plurality ofvirtual images is displayed at the same time, and the like, andnecessary data of the virtual image(s) is each acquired.

As the layout of the virtual image, for example, layout parameters suchas display position, size, and posture (tilt) of the virtual image arecalculated. As will be described later, the virtual image includes animage to be displayed using a coordinate system (body coordinate system)using the user 1 as the basis, an image to be displayed using acoordinate system (space coordinate system) using the surrounding space(space map or the like) as the basis, and the like. The layoutcalculation unit 26 calculates each of the display position, size, etc.of the virtual image in each coordinate system in accordance with theposture, position, etc. of the HMD 100 (user 1), for example.

In this embodiment, the layout calculation unit 26 controls the displayposition of the virtual image on the basis of the physical informationof the user 1. For example, as to a certain virtual image, after adisplay position corresponding to the posture and position of the HMD100 (user 1) is calculated, a display position (coordinate position) ofthe virtual image is, in turn, adjusted in accordance with the posture,height, and the like of the user 1. Moreover, in this embodiment, thelayout calculation unit 26 controls the display position of the virtualimage on the basis of the environmental information. For example, thedisplay position set on the basis of the physical information isadjusted in accordance with the presence/absence of a ceiling in thesurrounding environment or the like.

The method of controlling the display position of the virtual image bythe layout calculation unit 26 is not limited. For example, the displayposition may be adjusted using only one of the physical information orthe environmental information or the display position may be adjustedusing both the physical information and the environmental information asdescribed above. It should be noted that along with the adjustment ofthe display position of the virtual image, other layout parameters suchas the size and posture of the virtual image are adjusted so that thevirtual image is properly displayed.

Moreover, as the method of displaying the virtual image, there can beexemplified a method of performing conflict determination between thespace map and the virtual image (display object) and moving content, aGUI, or the like to an empty area of the real space. In this method,displaying can be performed so that a real object present in the realspace and a virtual image do not unnaturally overlap each other.Accordingly, displaying can be performed as if the virtual image werefloating in the real space. In addition to such display control, forexample, the layout calculation unit 26 performs display control usingthe above-mentioned physical information and environmental information.Accordingly, user-friendly virtual experience having a highentertainment property can be realized. The specific kind of the virtualimage and the control processing for the display position will bedescribed later in detail.

The layout determination unit 27 determines the layout of the virtualimage to be displayed on the display unit 10 in accordance with adegree-of-priority of the virtual image. For example, there is a casewhere a virtual image that is a display target is a display foremergency, such as an error message and a caution message, i.e., adisplay having a high degree-of-priority. In such a case, the layoutdetermination unit 27 generates a command to display the virtual imagethat is the target with a high degree-of-priority and outputs it to thelayout calculation unit 26. Accordingly, the virtual image for emergencyor the like can be reliably displayed.

The output control unit 28 generates an image to be output that includesthe virtual image displayed on the display unit 10. For example, on thebasis of the layout parameters calculated by the layout calculation unit26, an image to be output including one or more virtual images isgenerated. At that time, brightness, contrast, and the like may beadjusted. The generated image to be output is output to the display unit10 and displayed in front of the eyes of the user 1. As a result, theuser 1 wearing the HMD 100 can visually recognize the virtual imagesuperimposed on the real space.

In this manner, in the HMD 100, the gesture detection unit 25, thelayout calculation unit 26, the layout determination unit 27, and theoutput control unit 28 realize display control of the virtual image thatis based on the gesture operation of the user 1 or the progress of thecontent. It should be noted that in a case where the trigger gesture isa trigger for a gesture operation for controlling audio output,vibration output, or the like, output corresponding to the triggergesture is controlled. In this embodiment, the gesture detection unit25, the layout calculation unit 26, the layout determination unit 27,and the output control unit 28 cooperate to thereby realize the controlunit.

[Virtual Image]

FIG. 3 is a schematic diagram for describing an example of the virtualimage. In the example shown in FIG. 3 , the user 1 and a building arereal objects that exist in the real space. Moreover, virtual images 30 ato 30 g are examples of virtual objects to be displayed to the user 1wearing the HMD 100 so that the user 1 wearing the HMD 100 can visuallyrecognize them. Those virtual objects (virtual images 30) function asgraphical user interfaces (GUIs) that present various types ofinformation to the user 1, for example. Hereinafter, virtual images tobe displayed on the HMD 100 (display unit 10) will be specificallydescribed with reference to FIG. 3 .

The display positions of the virtual images 30 are set in accordancewith one or both of a body coordinate system using the user 1 (HMD 100)as the basis and a space coordinate system using the real space(surrounding environment) where the user 1 is present as the basis. InFIG. 3 , the body coordinate system and the space coordinate system areschematically shown with the solid-line arrow and the dotted-linearrows, respectively.

The body coordinate system is, for example, a coordinate systemfollowing position and posture of the HMD 100, i.e., position andposture of the head of the user 1. The space coordinate system is acoordinate system (world coordinate system) set in accordance with thespace map or the like irrespective of the position and posture of theHMD 100 and the like. Thus, when the user 1 wearing the HMD 100 moves,the point-of-origin of the body coordinate system as viewed from thespace coordinate system changes. Moreover, when the user 1 moves thehead, the orientation of the body coordinate system as viewed from thespace coordinate system changes.

The virtual image 30 a is, for example, an image that is arranged usingthe body coordinate system as the basis and is displayed to fall withinthe field-of-view of the user 1 irrespective of the posture of the headof the user 1 and the like. For example, even when the user 1 moves thehead, the user 1 can constantly visually recognize the virtual image 30a. The virtual image 30 a is, for example, used for displaying maincontent, an error message, or the like.

The virtual image 30 b is, for example, an image that is arranged usingthe body coordinate system and a vertical direction (upper-lowerdirection in the real space) as the basis and displayed on a diagonallyupper side with respect to the user 1. For example, by using the currentposition of the user 1 as the basis, the virtual image 30 b is arrangedalong an annular orbit set above the user 1. As shown in FIG. 3 , whenthe user 1 in an upstanding posture moves the head upward, the virtualimage 30 b is displayed on the display unit 10. Moreover, when the user1 looks forward again, the virtual image 30 b gets out of thefield-of-view (angle-of-view) of the user 1 and is not displayed. Thevirtual image 30 b is, for example, used for displaying a current time,the remaining battery of the HMD 100, status information such as acommunication status, a message, or the like. In this embodiment, thevirtual image 30 b corresponds to an upper image to be displayed on adiagonally upper side as viewed from the user.

The virtual image 30 c is, for example, an image that is arranged usingthe body coordinate system as the basis and displayed in an area aroundthe body (around the waist) of the user 1, i.e., a working space inwhich the user 1 can reach the hands. For example, in accordance with aselection operation (operation via a gesture or the operation inputunit) of the user 1, a plurality of virtual images 30 c is displayedcentered at the abdomen of the user 1. The virtual images 30 c are, forexample, used for displaying an operation selection screen (launcher)and main content.

The virtual image 30 d is, for example, an image that is arranged usingthe body coordinate system and the vertical direction as the basis anddisplayed near the feet of the user 1. Here, the virtual image 30 d isdisplayed along the floor (ground) of the place where the user 1 islocated. For example, the virtual image 30 d is displayed to move alongwith the movement of the user 1. Therefore, the virtual image 30 d isarranged using the position of the user 1 (body coordinate system) andthe floor (space coordinate system) as the basis. The virtual image 30 dis used for displaying a launcher, a navigation map, or the like. Inthis embodiment, the virtual image 30 d corresponds to a lower image tobe displayed on a diagonally lower side as viewed from the user.

The virtual image 30 e is, for example, arranged using the position ofthe user 1 and the space coordinate system as the basis and localizedand displayed in the real space so as to maintain a position relative tothe user 1. For example, when the user 1 moves, the position of thevirtual image 30 e in the space coordinate system also changes, whileeven when the user 1 changes the orientation, the position of thevirtual image 30 e does not change. The virtual image 30 e is, forexample, used for displaying a navigation object (e.g., arrow, agentcharacter) or the like.

The virtual image 30 f is, for example, an image that is arranged usingthe space coordinate system as the basis and displayed at constantly thesame size (field-of-view angle) irrespective of the distance from theuser 1. The virtual image 30 f is, for example, localized and displayedat a predetermined position of the space coordinate system. Both in acase where the display position of the virtual image 30 f is close and acase where the display position of the virtual image 30 f is far, theuser 1 visually recognizes the virtual image 30 f displayed with asimilar size. The virtual image 30 f is used for displaying, forexample, an icon, a message, or the like that displays information setto a real object (e.g., item, store, place).

The virtual image 30 g is, for example, an image that is arranged usingthe space coordinate system as the basis and displayed as if it werepresent in the real space. For example, as the distance between thedisplay position of the virtual image 30 g and the user 1 becomesshorter, the virtual image 30 g is displayed to be larger. That is, thevirtual image 30 g is an image that displays a superimposed object(virtual object) fixed in the space coordinate system.

It should be noted that the above-mentioned virtual images 30 a to 30 gare merely examples, and for example, arbitrary virtual images 30depending on content may be used. In addition, the kinds, applications,and the like of the virtual images 30 are not limited. Moreover, inarbitrary virtual images 30 having features as described above, thepresent technology can be applied.

Here, an example of display processing of the virtual image 30 thatcorresponds to a trigger gesture will be described. In the HMD 100, whena predetermined trigger gesture is detected, display processing of avirtual image 30 that corresponds to the detected trigger gesture isstarted. That is, the trigger gesture functions as a trigger for adisplay operation of causing the virtual image 30 to be displayed.Accordingly, only by performing a predetermined trigger gesture, theuser 1 can cause a desired virtual image 30 to be displayed.

In this embodiment, by the user 1 moving the head upward, the virtualimage 30 b (e.g., status) is displayed on the diagonally upper side asviewed from the user 1. In this case, the trigger gesture is a headgesture of the user 1 moving the head upward. It should be noted thatall gestures of the user 1 moving the head upward are not triggergestures, and in a case where an angle or the like of the posture of thehead of the user 1 exceeds a determination threshold, the head gestureis detected as the trigger gesture.

Moreover, by the user 1 moving the head downward, the virtual image 30 d(e.g., navigation map) is displayed on the diagonally lower side asviewed from the user. In this case, the trigger gesture is a headgesture of the user 1 moving the head downward. It should be noted thatprocessing of displaying the virtual image 30 c instead of the virtualimage 30 d or processing of determining a posture angle of the head anddisplaying each of the virtual images 30 c and 30 d can also beperformed. In addition, the types and the like of gestures that aretriggers for display operations are not limited, and the displayoperations may be performed in accordance with gestures of rotating thehead leftward and rightward, gestures of moving the arms or legs, or thelike.

Alternatively, the trigger gesture may be a trigger for a non-displayoperation of causing the virtual image 30 not to be displayed. Forexample, the above-mentioned head gesture of the user 1 moving the headupward may enable a gesture operation to hide the virtual image 30(e.g., virtual image 30 a shown in FIG. 4 ) displayed in front of theeyes of the user 1 to be performed. Alternatively, the head gesture ofthe user 1 moving the head downward may function as the trigger for thenon-display operation. As described above, the present technology canalso be applied in a case where the trigger for the non-displayoperation is used.

[Basic Operation of HMD]

FIG. 4 is a flowchart showing an example of a basic operation of the HMD100. The flowchart shown in FIG. 4 is, for example, processing performedrepeatedly when the HMD 100 is in the on-state. Hereinafter, taking acase where the user 1 performs a gesture operation (display operation)to operate display/non-display of the virtual image 30 as an example,the basic operation of the HMD 100 will be described.

First of all, the physical information acquisition unit 21 acquiresphysical information of the user 1 (Step 101). For example, on the basisof the output of the 9-axis sensor 17 or the like, a body posture (e.g.,standing state, seated state), a head posture, a limb state, and thelike of the user 1 are detected. Alternatively, for example, userinformation (height, gender, age, and the like) stored in the storageunit 15 is read in.

The environmental information acquisition unit 23 acquires environmentalinformation of the surrounding environment of the user 1 (Step 102). Forexample, depth information (depth data) of the periphery, an environmentshape, a space map, and the like are read in. Moreover, objectrecognition processing with respect to a real object(s) present in thesurrounding environment is performed, and for example, informationregarding the presence/absence of a ceiling at a current position of theuser 1, heights of ceiling and floor, and the like are acquired.Alternatively, for example, on the basis of a processing result of theobject recognition processing, the presence/absence of a movingobject(s) and the like are detected as the environmental information. Asdescribed above, it can also be said that the environmental informationis information indicating a recognition result of the externalenvironment.

The determination condition setting unit 24 sets a determinationcondition (determination threshold) for determining the trigger gesture(Step 103). The determination condition is set on the basis of thephysical information acquired in the above-mentioned step. For example,the determination condition appropriate to physical feature informationin a posture state (seated state or standing state) of the user 1 isset. Moreover, a threshold may be set on the basis of the environmentalinformation. The determination condition is changed, for example,depending on the presence/absence of the ceiling or the like. This pointwill be specifically described later. It should be noted that in a casewhere a plurality of trigger gestures is set, determination conditionscorresponding to the respective trigger gestures are each set on thebasis of the physical information or the environmental information.

Using the determination condition set in Step 103, the trigger gestureis detected (Step 104). First of all, the gesture informationacquisition unit 22 acquires gesture information (e.g., acceleration,angular velocity, and direction and the like of the HMD 100). Then, thegesture detection unit 25 determines whether or not the gestureindicated by the gesture information is the trigger gesture, using thedetermination condition.

For example, a threshold angle with respect to a posture angle (e.g.,pitch angle to be described later) of the head (HMD 100) is used as thedetermination condition of the trigger gesture of moving the head. Inthis case, the gesture detection unit 25 determines whether or not theposture angle of the head is larger than the threshold angle. In a casewhere the posture angle is larger than the threshold angle, it isdetermined that the trigger gesture is performed. In a case where theposture angle is smaller than the threshold angle, it is determined thatthe trigger gesture is not performed. This determination result (i.e.,detection result of the trigger gesture) is output to the layoutcalculation unit 26.

The layout calculation unit 26 acquires content information fordisplaying virtual images 30 (Step 105). Specifically, image data ofvirtual images 30 to be arranged in the content being executed in theHMD 100 is read in as the content information. Moreover, layoutparameters of the position and size and the like of each virtual image30 are calculated as the content information.

For example, in Step 104, in a case where a trigger gesture for adisplay operation of causing a virtual image 30 to be displayed isdetected, content information (data of the virtual image 30)corresponding to the trigger gesture is read in. It should be noted thatin a case where the trigger gesture is not detected, processing relatedto the display operation (reading the content information or the like)is not performed. Moreover, in a case where other content or the like isbeing executed, content information regarding the content is acquired asappropriate.

Moreover, the posture and position of the head of the user 1 (HMD 100)are detected, and the layout parameters of the virtual images 30 arecalculated in accordance with the detection result. For example, displaypositions, the sizes, and the like of a character, a message, and thelike included in the field-of-view of the user 1 are calculated asappropriate. It should be noted that in Step 105, for example, layoutparameters on which the physical feature of the user 1 and the like arenot reflected are calculated. As will be described later, those layoutparameters can be adjusted on the basis of the physical information andthe environmental information.

The layout determination unit 27 determines whether or not the virtualimage 30 is a display object set to have a high degree-of-priority(high-priority display object) (Step 106). Here, for example, thedegrees-of-priority of the respective virtual images 30 (contentinformation) acquired in Step 105 are each determined. For example, ahigh degree-of-priority is set to a virtual image 30 that presentsemergency information such as an error message and a caution message. Incontrast, a low degree-of-priority is set to a normal virtual image 30that displays a status, a menu screen, or the like. Alternatively, thedegree-of-priority may be set only to the virtual image 30 for emergencyand the degree-of-priority does not need to be set to the normal virtualimage 30.

In a case where it is determined that the virtual image 30 has a highdegree-of-priority (YES in Step 106), display with initial values isselected (Step 107). For example, a command signal indicating that thevirtual image 30 having a high degree-of-priority is to be displayedwith pre-set layout parameters is generated. Accordingly, it is possibleto, for example, constantly display the virtual image 30 having a highdegree-of-priority in the middle of the field-of-view of the user 1 orthe like, and an urgent message or the like can be reliably presented tothe user 1.

In a case where it is determined that the virtual image 30 has a lowdegree-of-priority (NO in Step 106), display corresponding to thephysical information or the environmental information is selected (Step108). For example, a command signal indicating that the virtual image 30having a low degree-of-priority is to be displayed with layoutparameters adjusted in accordance with the physical information or theenvironmental information is generated. That is, as to the virtual image30 having a low degree-of-priority, processing of controlling thedisplay position in accordance with the posture of the user 1, thepresence/absence of the ceiling in the space where the user 1 ispresent, or the like is selected. Accordingly, for example, it ispossible to display the virtual image 30 at a position easy for the user1 to visually recognize, and a user-friendly GUI can be realized.

In this manner, in accordance with the degree-of-priority of the virtualimage 30 (GUI or the like) to be presented to the user 1, whether toperform displaying in view of the physical feature is determined. Forexample, there can be a case where it is unfavorable to adjust thedisplay position and the like of the virtual image 30 having a highdegree-of-priority like the above-mentioned error message or the like.In this embodiment, by performing the determination processing in Step106, virtual images 30 whose display positions should be adjusted andvirtual images 30 that should be displayed without changing thepositions can be distinguished from each other and processed.Accordingly, the HMD 100 easy to use and highly reliable can berealized.

In accordance with the display method selected in Steps 107 and 108,output processing of the virtual images 30 is performed (Step 109).First of all, the layout calculation unit 26 adjusts the layoutparameters for each virtual image 30. As to the virtual image 30 havinga low degree-of-priority, the display position is adjusted in accordancewith the physical information or the environmental information. As tothe virtual image 30 having a high degree-of-priority, a default displayposition is set. Then, the output control unit 28 generates an image tobe output on the basis of the layout parameters of the virtual images 30and outputs it to the display unit 10. In this manner, in the HMD 100,the virtual images 30 such as GUIs are displayed on the basis of thedetermination conditions and the display positions set in accordancewith the physical information and the environmental information.

[Threshold Setting and Display Control]

Hereinafter, taking a gesture operation by a head gesture of the user 1moving the head upward (hereinafter, referred to as upward gesture) asan example, setting processing of the determination condition(determination threshold) and display control of the virtual image 30will be described specifically. It should be noted that as describedabove with reference to FIG. 3 , the upward gesture functions as thetrigger gesture for the display operation of causing a predeterminedvirtual image 30 (virtual image 30 b) to be displayed.

First of all, the posture angle of the head will be described. Theposture angle of the head is, for example, represented by a pitch angle,a roll angle, and a yaw angle. Those posture angles are, for example,angles using a horizontal plane and a vertical direction as the basis.

The pitch angle is an angle using a left-right direction of the head asthe rotational axis, and represents, for example, an angle of tilt (flexangle or extension angle) of the head when the neck is tilted forward orrearward. The roll angle is an angle using a front-back direction of thehead as the rotational axis, and represents, for example, an angle oftilt of the head (lateral flex angle) when the neck is tilted leftwardor rightward. The yaw angle is an angle having an upper-lower directionof the head as the rotational axis, and represents, for example, anangle of rotation (rotational angle) of the head when the neck isrotated leftward or rightward. It should be noted that other than thecase of moving the neck, for example, when the user 1 bends the upperpart of the body forward or backward, leftward or rightward, the pitchangle and the roll angle change, and when the user 1 rotates the upperpart of the body, the yaw angle changes.

In this embodiment, the posture angle of the head of the user 1 isrepresented by the posture angle (e.g., pitch angle, roll angle, yawangle) of the HMD 100. The posture angle of the HMD 100 is, for example,detected on the basis of the output of the 3-axis gyro sensor or thelike. It should be noted that in a case where the external camera or thelike is provided, the posture angle of the head may be directlyestimated by performing skeleton estimation or the like on the basis ofa taken image of the user 1. In addition, the method of calculating theposture angle of the head and the like are not limited. Hereinafter, itis assumed that a state (reference posture) in which the head of theuser 1 faces forward, each posture angle is 0°. Moreover, the postureangle of the HMD 100 may be referred to as the posture angle of thehead.

In a case where the upward gesture is performed, the pitch angle of thehead (HMD 100) changes upward due to the user 1 moving the head upward.A threshold angle (upper threshold angle) with respect to this pitchangle that changes upward is set as the determination condition fordetermining the upward gesture as the trigger gesture. That is, thedetermination condition for the upward gesture includes the upperthreshold angle with respect to the pitch angle of the head of the user1 that changes due to the upward gesture. It should be noted that theupper threshold angle is, for example, set as an angle using the pitchangle=0° as the basis.

For example, in a case where the pitch angle does not exceed the upperthreshold angle in the motion of the user 1 moving the head upward, itis determined that the trigger gesture is not performed. Otherwise, in acase where a motion with the pitch angle exceeding the upper thresholdangle is performed, the motion (upward gesture) of the user 1 isdetermined as the trigger gesture. Then, in a case where it isdetermined that the trigger gesture is performed, the virtual image 30is displayed on the diagonally upper side of the user 1.

FIG. 5 is a schematic diagram showing an example of the virtual image 30displayed by the upward gesture. FIG. 5 schematically shows an exampleof the virtual image 30 (upper image 31) displayed on the diagonallyupper side of the user 1 by the user 1 performing the upward gesture. Itshould be noted that the upper image 31 is a specific display example ofthe virtual image 30 b described above with reference to FIG. 3 .

As shown in FIG. 5 , the upper image 31 includes, for example, a statusicon 32, a time display 33, a date display 34, and a messagenotification 35. The status icon 32 includes an icon showing acommunication status of the HMD 100 (e.g., connection status with thenetwork such as Wi-Fi), an icon showing the remaining battery of the HMD100, and the like. The time display 33 and the date display 34 displaycurrent time and date. The message notification 35 is a textnotification that notifies newly arrived e-mail, notifications, and thelike. In addition, the specific displayed contents of the upper image 31are not limited.

When the user 1 who uses the HMD 100 looks forward, the upper image 31as shown in FIG. 5 is not displayed, and therefore a wider field-of-viewis ensured. For example, for checking the status and the like, the user1 moves the head upward by some angle (upper threshold angle), tothereby start display processing of the upper image 31 and present thestatus or the like. Moreover, the upper image 31 is localized andarranged above the user 1, and therefore when the user 1 looks forwardagain, it is not displayed. It should be noted that the upper image 31is retained as active content for a certain time. Thus, when the user 1moves the head upward while the upper image 31 is active, the upperimage 31 is displayed irrespective of the upward gesture determinationand the like.

[Features of Vertical Field-of-View]

FIG. 6 is a schematic diagram showing an example of features relating toa human vertical field-of-view. FIG. 6 has angles (angle ranges) showingrespective features of the vertical field-of-view using a case where thehuman head faces forward as the basis. Each of the features shown inFIG. 6 is represented by an angle using, as the basis, a line-of-sightdirection (reference line-of-sight 40) when a person looks in thehorizontal direction in the state in which the human head faces forward.

A vertical movement limit of the human eye, i.e., a vertical maximum eyemovement range is about 30° on the upper side and about 35° on the lowerside. Moreover, a range in which the person can easily move the eyes isa range of about 30° on the lower side of the reference line-of-sight40. The maximum eye movement range is a variable range in which theperson can change the line-of-sight direction without changing theorientation of the head and the like. An easy eye movement range is arange in which the person can easily change the line-of-sight direction.

The field-of-view when the human line-of-sight is the referenceline-of-sight 40 (when the person looks in the horizontal direction) isabout 50° on the upper side and about 70° on the lower side. In thisfield-of-view, a range (limit of color discrimination) in which theperson can discriminate the color is about 35° on the upper side andabout 40° on the lower side. It should be noted that the field-of-viewrange and the limit of color discrimination change in accordance withthe line-of-sight direction (eye orientation).

Moreover, in FIG. 6 , a human normal line-of-sight direction (normalline-of-sight 41) is shown. In general, the normal line-of-sight 41changes depending on the human posture. For example, in a seated statein which the person is seated, the normal line-of-sight 41 is adirection of about 15° on the lower side of the reference line-of-sight40. Moreover, in a standing state in which the person is standing, thenormal line-of-sight 41 is a direction of about 10° on the lower side ofthe reference line-of-sight 40. Thus, in the seated state, the direction(normal line-of-sight 41) in which the person normally looks is lowerthan that in the standing state by about 5°.

In this manner, the normal line-of-sight 41 of the user 1 (person),i.e., a range that the user 1 normally visually recognizes changesdepending on the posture of the user 1. In this embodiment, inaccordance with such a field-of-view feature in the posture of the user1, the determination condition (the upper threshold angle of the upwardgesture) is set.

[Threshold Setting According to Posture]

FIGS. 7A and 7B are schematic diagrams for describing setting of thedetermination condition appropriate to the posture of the user 1. Inthis embodiment, the determination condition setting unit 24 sets thedetermination condition in accordance with the posture of the user 1.For example, the posture information indicating whether or the user 1 isin the seated state or the standing state is acquired in Step 101 ofFIG. 4 . Then, the upper threshold angle that is the determinationcondition for the upward gesture is set using the posture information inStep 103. FIG. 7A and FIG. 7B schematically show the user 1 in thestanding state and the user 1 in the seated state, respectively.

An upper-limit threshold angle α for determining the upward gesture is,for example, set in accordance with the posture of the user 1 by using apre-set reference threshold angle α₀ as the basis. This referencethreshold angle α₀ is, for example, a threshold angle set by using, asthe basis, the state (state of the reference line-of-sight 40) in whichthe user 1 looks in the horizontal direction.

For example, the reference threshold angle α₀ is set to be a maximumvalue of 30° on the upper side that is the movement limit of the eyeshown in FIG. 6 (α₀=30°). It corresponds to the upper-limit angle in thevertical direction at which the user 1 in the reference posture can gazeby moving the eyes without moving the head. That is, it can also be saidthat a minimum pitch angle that the user 1 moves in order to capture, inthe front, a display object, at which the user 1 cannot gaze even bymoving the eyes, is set as the reference threshold angle α₀. The methodof setting the reference threshold angle α₀ is not limited, and forexample, the reference threshold angle α₀ may be set in accordance withthe limit of color discrimination, the visual limit, or the like.

In this embodiment, the upper threshold angle is set by subtracting theangle θ of the normal line-of-sight 41 in the posture of the user 1 fromthe reference threshold angle α₀. That is, the upper threshold angle αis set to α=α₀−θ. For example, as shown in FIG. 7A, when the user 1 isin the standing state, the upper threshold angle is set to α=α₀−10°.Moreover, as shown in FIG. 7B, when the user 1 is in the seated state,the upper threshold angle is set α=α₀−15°. For example, where α₀=30°,the upper threshold angle is set to α=20° in the standing state and theupper threshold angle is set to α=15° in the seated state. In thismanner, the determination condition setting unit 24 sets, in a casewhere the seated state is detected, the threshold angle to be a smallervalue than in a case where the standing state is detected.

For example, in the state in which the user 1 is seated, an averageline-of-sight (normal line-of-sight) of the user 1 is directed closer(lower) as compared with the standing state. Therefore, it isconceivable that even when the user 1 has intended to perform a similargesture in a case of performing the display operation by the upwardgesture, the pitch angle in moving the head upward is smaller in theseated state than in the standing state unconsciously.

In contrast, in this embodiment, by setting the upper threshold angle αin the seated state to be lower than in the standing state, the user 1can display the upper image 31 by a natural motion irrespective of theposture of the user 1. Moreover, in the standing state, the upperthreshold angle α is set to be larger than in the seated state, andtherefore determination errors and the like due to the fact that thepitch angle in performing the upward gesture is relatively larger can bereduced. Accordingly, operability of virtual experience using gestureoperations and the like can be sufficiently improved.

It should be noted that the determination condition of the head gestureof the user 1 moving the head downward (hereinafter, referred to asdownward gesture) may be set utilizing the difference of the normalline-of-sight 41 that depends on the posture of the user 1. For example,the downward gesture functions as a trigger gesture for displaying thevirtual image 30 d (lower image) shown in FIG. 3 . As the determinationcondition of this downward gesture, a lower threshold angle β withrespect to the pitch angle that changes downward is set.

The determination condition setting unit 24 sets the lower thresholdangle β by, for example, subtracting the angle of the normalline-of-sight in the seated state or the standing state from a referencethreshold angle β₀. The reference threshold angle β₀ is, for example,set to be a maximum value of 35° that is the movement limit of the eyeon the lower side or the like. Accordingly, irrespective of the postureof the user 1, the user 1 can display the lower image by a naturalmotion. For example, as described above, the determination condition maybe set.

[Display Control According to Posture]

Moreover, in this embodiment, the layout calculation unit 26 controlsthe display positions of the virtual images 30 in accordance with theposture of the user 1. For example, the vertical display position of theupper image 31 to be displayed in accordance with the upward gesture isadjusted in accordance with the posture (standing state or seated state)of the user 1. In FIG. 7A and FIG. 7B, the upper images 31 displayedwhen the user 1 is in the standing state and when the user 1 is in theseated state are schematically shown as the hashed regions.

The upper image 31 is arranged to fall within the field-of-view of theuser 1 in a state in which the pitch angle of the head of the user 1 hasreached the upper threshold angle α, for example. For example, thedisplay position of the upper image 31 is adjusted so that the upperimage 31 is displayed in a predetermined region within the field-of-viewinclined at the upper threshold angle α (e.g., a region of the upperhalf of the field-of-view or the like). In this case, as long as theposture of the user 1 is not changed and the pitch angle is the same,the upper image 31 is displayed in a constant position within thefield-of-view of the user 1. It should be noted that when the posture ofthe user 1 is changed, the display position of the upper image 31(height position or the like as viewed from the head) differs.

For example, as shown in FIG. 7A, in the standing state, in accordancewith the upper threshold angle α set to be larger than in the seatedstate, the upper image 31 is arranged at a relatively high position asviewed from the head of the user 1. Moreover, as shown in FIG. 7B, inthe seated state, the upper threshold angle α is set to be a smallervalue than in the standing state, and therefore the upper image 31 isarranged at a lower position in the standing state as viewed from thehead of the user 1. In this manner, the layout calculation unit 26 setsthe display position of the upper image 31 to be a lower position in acase where the seated state is detected than in a case where thestanding state is detected.

Accordingly, irrespective of the posture of the user 1, the user 1 canvisually recognize the upper image 31 smoothly as the extension of theupward gesture. Moreover, a gesture that the standing user performs canhave, for example, an amount of motion larger than in the case where theuser is seated or the like. In such a case, the upper image 31 can bedisplayed at a suitable position corresponding to the amount of motionof the user 1. Moreover, the seated user 1 can display the upper image31 by an upward gesture having a small amount of motion and can easilyvisually recognize the upper image 31 displayed at a relatively lowposition at the same time.

It should be noted that the method of setting the display position ofthe upper image 31 is not limited. For example, for a case where theuser 1 is standing and a case where the user 1 is seated, positions easyfor the user 1 to visually recognize may be pre-set as default positionsand the upper image 31 may be arranged at each default position in eachposture of the user 1. Moreover, other than the display position, otherlayout parameters such as the size and posture of the upper image 31 maybe adjusted in accordance with the posture.

[Threshold Setting According to Environmental Information]

Hereinabove, the setting of the determination condition and the likeappropriate to the posture of the user 1 have been described. In the HMD100, the determination condition setting unit 24 sets the determinationcondition on the basis of the environmental information. Accordingly,for example, the determination condition appropriate to the physicalfeature information in the surrounding environment in addition to thephysical feature information in the posture state of the user 1 can beset.

The normal line-of-sight 41 (average line-of-sight) described above withreference to FIG. 6 can change depending on the surrounding environmentof the person. For example, in a case where the person is outdoor, thenormal line-of-sight 41 tends to be closer to the horizontal directionas compared with a case where the person is indoor. Moreover, even inthe case where the person is indoor, as the ceiling is at a higherlevel, the normal line-of-sight 41 tends to be closer to the horizontaldirection. Hereinafter, an angle of change of the normal line-of-sight41 that changes depending on the environment will be referred to as anenvironment angle ϕ. The environment angle β is, for example, is anangle of 10° or less, and changes depending on the height of the ceilingand the like.

For example, it is assumed that the normal line-of-sight 41 in a casewhere the user 1 is indoor in the standing state is 10°. In a case wherethis user 1 is outdoor in the standing state, the normal line-of-sight41 tends to be higher (e.g., 5° or the like). In this case, assumingthat the feature of the normal line-of-sight 41 shown in FIG. 6 is areference value, the environment angle ϕ in the indoor environment is 0°and the environment angle ϕ in the outdoor environment is 5°. It shouldbe noted that in a case where the normal line-of-sight 41 is lower thanthe reference value, for example, the environment angle ϕ can also takea negative value. In practice, the environment angle ϕ is set on thebasis of the normal lines-of-sight 41 and the like actually measured inthe indoor environment and outdoor. Alternatively, the environment angleϕ may be set by learning the motion tendence of the user 1.

In the HMD 100, the upper threshold angle for determining the upwardgesture is adjusted in accordance with the feature difference of thehuman vertical field-of-view (change in the environment angle ϕ) due tosuch a difference of the surrounding environment. Hereinafter, the upperthreshold angle set in accordance with the surrounding environment willbe referred to as an upper threshold angle α′.

First of all, the environmental information regarding the surroundingenvironment of the user 1 is acquired (in Step 102 of FIG. 4 ). In thisembodiment, the environmental information acquisition unit 23 detectsthe presence/absence of the ceiling in the surrounding environment. Thepresence/absence of the ceiling is, for example, detected on the basisof a space map generated by the use of the SLAM or the like and an imagetaken by the rear-facing camera 16. The detection result of thepresence/absence of the ceiling is output to the determination conditionsetting unit 24. Then, upper threshold angles α′ depending on thesurrounding environments respectively are calculated for a case wherethe ceiling is present and a case where the ceiling is absent.

In the determination condition setting unit 24, for example, by addingthe environment angle □ to the upper threshold angle depending on theposture of the user 1 (standing state or seated state), which has beendescribed above with reference to FIGS. 7A and 7B, the upper thresholdangle □′ depending on the surrounding environment is calculated(□′=□+□). It corresponds to, for example, subtracting the angle of thenormal line-of-sight 41 in each posture in the indoor environment or theoutdoor environment from the reference threshold angle □0, which hasbeen described above.

For example, in a case where the ceiling is detected, considering thatthe user 1 is indoor, the upper threshold angle α′ depending on thesurrounding environment is set to be the value obtained by adding theenvironment angle ϕ (e.g., 0°) in the indoor environment to the upperthreshold angle α depending on the posture. On the other hand, in a casewhere the ceiling is not detected, considering that the user 1 isoutdoor, the upper threshold angle α′ is set to be the value obtained byadding the environment angle ϕ (e.g., 5°) in the outdoor environment tothe upper threshold angle α.

In this manner, the determination condition setting unit 24 sets, in acase where the ceiling is detected, the upper threshold angle α′ to be asmaller value than in a case where the ceiling is not detected.Accordingly, both in the case where the current position is indoor andin the case where the current position is outdoor, the user 1 can easilydisplay the upper image 31 by naturally performing the upward gesture.As a result, the HMD 100 that provides excellent operability in variousfields can be realized.

It should be noted that as described above, the present technology isnot limited to the case where the upper threshold angle α′ is set inaccordance with the presence/absence of the ceiling, and for example,the upper threshold angle α′ may be set in accordance with the height ofthe ceiling. For example, as the height of the ceiling becomes higher,the human normal line-of-sight 41 tends to be closer to the horizontaldirection. In accordance with such a feature, for example, as the heightof the ceiling becomes higher, the upper threshold angle α′ is set to belarger. Accordingly, the gesture determination based on thefield-of-view feature that changes depending on the difference of theheight of the ceiling can be performed, and the operability can beimproved.

[Display Control According to Environmental Information] FIGS. 8A and 8Bare schematic diagrams showing a display example of the upper image 31corresponding to the environmental information. In FIGS. 8A and 8B, inthe indoor environment where the ceiling is present and the outdoorenvironment where the ceiling is absent, the upper images 31superimposed on the field-of-view of the user 1 are schematically shown.It should be noted that the field-of-view of the user 1 shown in FIGS.8A and 8B is a field-of-view with the same posture of the user 1, thesame height of the head from the floor, and the same pitch angle.

In this embodiment, the layout calculation unit 26 adjusts the displayposition of the upper image 31 in accordance with the presence/absenceof the ceiling. For example, in a case where the user 1 is indoor (FIG.8A), the upper image 31 is displayed at a lower position as comparedwith a case where the user 1 is outdoor (FIG. 8B). Those displaypositions can be set by using, for example, the upper threshold anglesα′ in the indoor and outdoor environments as the basis. Accordingly, theuser 1 can visually recognize the upper image 31 smoothly as theextension of the upward gesture irrespective of the indoor/outdoorenvironment.

In the indoor environment, a target of interest of the user 1 is oftenin a space between the floor and the ceiling. Thus, even in a case wherethe display position of the upper image 31 is slightly lower as shown inFIG. 8A, the situation where the upper image 31 overlap a region atwhich the user 1 wishes to gaze can be sufficiently avoided. Moreover,the status and the like of the upper image 31 can be checked withouttilting the head unnecessarily, and therefore the burden on the body ofthe user 1 can be reduced.

Moreover, in the outdoor environment, the user 1 can direct theline-of-sight to a further position. Therefore, as shown in FIG. 8B, bysetting the display position of the upper image 31 to be higher, thefield-of-view of the user 1 can be widely ensured, and the visibility tothe real space can be increased. Moreover, the user 1 located in theoutdoor environment can naturally perform even a relatively largemotion. Therefore, even in a case where the display position of theupper image 31 is slightly higher, the user 1 can visually recognize theupper image 31 easily by a natural motion.

[Features of Horizontal Field-of-View]

FIG. 9 is a schematic diagram showing an example of features relating toa human horizontal field-of-view. FIG. 9 schematically shows a person asviewed from the top of the head. FIG. 9 has angles (angle ranges)showing respective features of the horizontal field-of-view using a casewhere the human head faces forward as the basis. The lower side in thefigure corresponds to the front of the user 1 and the upper side isequivalent to the back of the user 1.

A horizontal movement limit of the human eye, i.e., a horizontal maximumeye movement range are about 15° on the left and right sides. Thefield-of-view of the left eye in a case (reference line-of-sight 40)where the human line-of-sight is directed forward is about 94° on theleft side and about 62° on the right side. Within this field-of-view,the limit of color discrimination in which the person can discriminatethe color is about 30° to 60° on the left and right side.

For example, by displaying the virtual image 30 to fall within thehorizontal maximum eye movement range, the user 1 can easily recognizethe virtual image 30. On the other hand, in general, the displaypositions of the virtual images 30 to be displayed in accordance withthe progress of the content and user operations (gesture operations orthe like), also including the above-mentioned upper images 31, can bearbitrarily set. That is, the virtual images 30 to be displayed are notnecessarily displayed at positions easy for the user 1 to visuallyrecognize. Therefore, for example, situations where the user 1 losestrack of the virtual image 30 and where the user 1 does not notice thedisplayed virtual image 30 may occur.

By the way, depending on the surrounding environment of the user 1, itmay be possible to estimate a region (direction) or the like easy forthe user 1 to visually recognize. In such a case, by setting the displayposition of the virtual image 30 to be in the region easy for the user 1to visually recognize, the situation where the user 1 loses track of thevirtual image 30 can be avoided. Hereinafter, display control based onthe surrounding environment will be described specifically.

FIG. 10 is a schematic diagram showing a display example of the virtualimage 30 based on the environmental information. In FIG. 10 , afield-of-view image 50 of the user 1 which has been taken by therear-facing camera 16 is schematically shown. The field-of-view image 50is an image of an indoor passageway and includes a single vanishingpoint 51 near the middle of the image. That is, the composition of thefield-of-view image 50 can be considered as single point perspective. Assuch, in a case where the field-of-view of the user 1 can be consideredas single point perspective, line-of-sight directions of the user 1easily concentrate the vicinity of the vanishing point 51 in thefield-of-view of the user 1. In other words, the vicinity of thevanishing point 51 is a region easy for the user 1 to visuallyrecognize.

In this embodiment, the display position is controlled so that thevirtual image 30 is displayed near the vanishing point 51. Specifically,the environmental information acquisition unit 23 determines whether ornot the surrounding environment includes the vanishing point 51. Then,in a case where the surrounding environment includes the vanishing point51, the layout calculation unit 26 sets the display position of thevirtual image 30 by using the vanishing point 51 as the basis.

The environmental information acquisition unit 23 determines thepresence/absence of the vanishing point 51 by, for example, detecting afeature amount of a straight line or the like from the field-of-viewimage 50 taken by the rear-facing camera 16. Alternatively, thevanishing point 51 may be detected by the use of three-dimensional datasuch as a space map, for example. Alternatively, in a case where thefront-facing camera or the like that takes images of the eyes of theuser 1 is provided, motions of the eyes of the user 1 may be detected.In this case, in an average motion of the eye is little (e.g., in a casewhere the motion of the line-of-sight of the user 1 is sufficientlysmaller than the horizontal movement limit or the like), it may bedetermined that the vanishing point 51 is present, considering thatsingle point perspective is used. In addition, the method of determiningthe presence/absence of the vanishing point 51 is not limited.

In a case where it is determined that the surrounding environmentincludes the vanishing point 51, the layout calculation unit 26 sets,for example, a display candidate region 52 (hashed rectangle in FIG. 10) using the vanishing point 51 as the basis. The display candidateregion 52 is set to have a lateral width of about 10°, for example, inthe horizontal field-of-view. The virtual image 30 is arranged to fallwithin the set display candidate region 52. Accordingly, for example, ina case where the user 1 is present in the surrounding environmentincluding the vanishing point 51 like a passageway, the virtual image 30is displayed in the vicinity of the vanishing point 51. Accordingly, thevirtual image 30 can be displayed at a position to which the user 1naturally directs the line-of-sight, and therefore the accessibility tothe virtual image 30 can be sufficiently improved.

Moreover, in this embodiment, the layout calculation unit 26 determineswhether or not to cause the virtual image 30 to fall within the displaycandidate region 52. Specifically, whether or not to cause it to fallwithin the display candidate region 52 is determined in accordance witha distance of a point of intersection (conflict point) between theline-of-sight of the user 1 and the surrounding environment convertedinto the space map. For example, in a case where the point ofintersection is present in a range up to a short distance (about 3 m) atwhich the user can view the virtual image 30 stereoscopically,processing of displaying the virtual image 30 as it is without causingthe virtual image 30 to fall within the display candidate region 52 isperformed. That is, in a case where the line-of-sight of the user 1 isdirected to a short distance, normal display processing is performed,and therefore the user 1 can easily recognize the virtual image 30.

[Features of Range-of-Motion]

FIG. 11 is a schematic diagram showing an example of features relatingto a human range-of-motion. FIG. 11 schematically shows, using a statein which the person looks forward as the reference posture, examples ofthe range-of-motion of the neck at the time of flex to tilt the headforward and the range-of-motion of the neck at the time of extension totilt the head downward. Easy flex angles at which the flex and extensioncan be easily performed are both about 30°. Moreover, maximum flexangles at which the flex and extension can be performed are both about60°. Those features relating to the range-of-motion of the neck areincluded in the motion feature according to this embodiment.

In this embodiment, the determination condition is set in accordancewith the range-of-motion of the user 1 in the posture of the user 1. Forexample, in the standing state in which the user 1 is standing, the easyflex angle and the maximum flex angle of the neck becomes larger ascompared with the seated state in which the user 1 is seated. Moreover,in the standing state, the angle of tilt of the head that the user 1 cantilt by bending the waist also becomes larger as compared with theseated state. In accordance with such a difference of motion features ofthe neck and the waist, the determination conditions (upper thresholdangle □ and lower threshold angle □) of the upward gesture and thedownward gesture are set, which have been described above with referenceto FIGS. 7A and 7B and the like.

The determination condition setting unit 24 sets, for example, referencethreshold angles α₀ and β₀ that are the reference for setting the upperthreshold angle α and the lower threshold angle β in accordance with theposture of the user 1. For example, in a case where the user 1 is in theseated state, the reference threshold angles α₀ and β₀ are set to theupper limit value and the lower limit value of the movement limit of theeye shown in FIG. 4 . Moreover, in a case where the user 1 is in thestanding state, the reference threshold angles α₀ and β₀ are set to belarger than values in the seated state. That is, in the standing state,ranges of threshold angles for determining the upward gesture and thedownward gesture are adjusted to be wider than in the seated state.

The amounts of increase of the reference threshold angles α₀ and β₀ inthe standing state are set in accordance with the amounts of increase ofthe range-of-motion (easy flex angle and maximum flex angle or the like)in the standing state as compared with the seated state, for example.That is, the reference threshold angles α₀ and β0 are set in accordancewith the movable range that extends when the user 1 stands up. It canalso be said that the ranges of the threshold angles α and β on theupper side and lower side are scaled in accordance with therange-of-motion. Accordingly, for example, each of the threshold anglesα and β can be set to be larger within a range in which the user 1 canmove comfortably. As a result, without increasing the burden on the bodyof the user 1, erroneous detection and the like of the upward anddownward gestures can be reduced.

It should be noted that the method of setting the threshold angle inaccordance with the range-of-motion and the like are not limited. Forexample, the reference threshold angles α₀ and β₀ in the seated statemay be set in accordance with the range-of-motion. Moreover, rather thansetting the reference threshold angles α₀ and β₀, an amount of shiftdepending on the range-of-motion may be set in each posture. By addingthis amount of shift to the threshold angles (α and β) set in accordancewith the posture information of the user 1 and the environmentalinformation, final threshold angles may be set. Moreover, the feature ofthe range-of-motion in each posture has individual differences, andtherefore for example, a configuration to learn the range-of-motion ofthe user 1 and set threshold angles on the basis of the learning resultmay be employed.

[Field-of-View Feature According to Height]

FIG. 12 is a schematic diagram showing an example of display positionsof virtual images 30 displayed on a diagonally lower side as viewed fromthe user 1. Examples of the virtual images 30 displayed on a diagonallylower side can include the virtual image 30 c and the virtual image 30 d(lower image 36) described above with reference to FIG. 3 . Hereinafter,referring to FIG. 12 , the method of setting the display position of thelower image 36 will be described.

In general, when the person looks diagonally down, a range in which anangle of depression γ is approximately 10° to 30° is a range in whichthe person can easily visually recognize. Here, the angle of depressionγ is an angle of looking displayed on a diagonally lower side from thehorizontal direction. Hereinafter, a region that falls within theabove-mentioned angle of depression and has a constant height will bereferred to as a lower display region 54. Moreover, the horizontal planeincluding the head of the user 1 (HMD 100) will be referred to as areference horizontal plane 55. In FIG. 12 , three kinds of lower displayregions 54 a to 54 c whose relative distances H from the referencehorizontal plane 55 are different are schematically shown as the hashedregions.

The lower display region 54 is a region whose horizontal distance fromthe user 1 is H/tan (10°) to H/tan (30°). As shown in FIG. 12 , thefront position of the lower display region 54 as viewed from the user 1becomes further from the user 1 as the relative distance H from thereference horizontal plane 55 (height position of the head of the user1) becomes longer. Moreover, the size (depth) of the lower displayregion 54 becomes larger as the relative distance H becomes longer. Inthis manner, in a case where the relative distance H differs, theposition and the size of the lower display region 54 change.

In this embodiment, the layout calculation unit 26 sets the displayposition of the lower image 36 so that the lower image 36 falls withinthe lower display region 54. Specifically, the relative distance H (theheight position of the virtual image 30) from the reference horizontalplane 55 at which the virtual image 30 is to be displayed is calculatedon the basis of the height of the user 1, and the depth position of thevirtual image 30 is set on the basis of the relative distance H. In thismanner, in this embodiment, the display position of the lower image 36is set in accordance with the height of the user 1.

FIGS. 13A and 13B are schematic diagrams showing an example of the lowerimage 36. FIG. 13A is an example of the lower image 36 displayed aroundthe body (around the waist) of the user 1 and is a display example ofthe virtual image 30 c shown in FIG. 3 . The virtual image 30 c is aselection screen including a plurality of selection windows 37. Forexample, the user 1 can perform a gesture operation to selecting adesired window 37 from those windows 37 by rotating the head leftwardand rightward. For example, such a virtual image 30 c is displayed atthe height of the abdomen of the user 1.

The virtual image 30 c is arranged in the lower display region 54 a setat a height approximately equivalent to that of the abdomen of the user1 shown in FIG. 12 . A relative distance H_(a) of the lower displayregion 54 a from the reference horizontal plane 55 is, for example, setto be the half of a height h₁ of the user 1 (H_(a)=h₁/2). Alternatively,for example, in a case where a height h₂ of the waist of the user 1 isread in or the like, a value obtained by subtracting the height h₂ ofthe waist from the height h₁ is set to the relative distance H_(a)(H_(a)=h₁−h₂). In addition, the method of setting the relative distanceH_(a) for displaying the virtual image 30 c is not limited.

When the relative distance H_(a) is set, a horizontal position of thelower display region 54 a is calculated, and the display position of thevirtual image 30 c is set to fall within the lower display region 54 a.It should be noted that the virtual image 30 c is an image to bedisplayed as if it were floating in the air, and is, for example,arranged to be tilted so that the user 1 can easily view the image.Accordingly, the user 1 can easily visually recognize the virtual image30 c displayed around the body of the user 1.

FIG. 13B is an example of the lower image 36 displayed on the floor ofthe periphery of the user 1 and is a display example the virtual image30 d shown in FIG. 3 . The virtual image 30 d is, for example, a screendisplaying the navigation map 38 and the like. Moreover, in the exampleshown in FIG. 13B, an operation window 39 for performing a navigationoperation is displayed, and the user 1 can perform various operations byperforming a predetermined selection operation. For example, such avirtual image 30 d is displayed on the floor.

The virtual image 30 d is arranged in a lower display region 54 b set atthe same height as the floor on which the user 1 is standing, which isshown in FIG. 12 . In this case, a relative distance H_(b) of the lowerdisplay region 54 b from the reference horizontal plane 55 is, forexample, set as the height h₁ of the user 1 (H_(b)=h₁). Alternatively,for example, a value obtained by subtracting an amount of deviation ofthe mounting position of the HMD 100 from the height h₁ of the user 1may be set as the relative distance H_(b). In addition, the method ofsetting the relative distance H_(b) for displaying the virtual image 30d is not limited.

When the relative distance H_(b) is set, a horizontal position of thelower display region 54 b is calculated, and the display position of thevirtual image 30 d is set to fall within the lower display region 54 b.The virtual image 30 d is, for example, an image to be displayed alongthe floor. Therefore, the user 1 can visually recognize the navigationmap and the like displayed using the floor of the periphery as thescreen. Moreover, the virtual image 30 d is displayed at a position thatthe user 1 can easily visually recognize, and therefore it is, forexample, unnecessary for the user 1 to unnecessarily bend the neck, andthe burden on the body of the user 1 can be sufficiently reduced.

The lower display region 54 c shown in FIG. 12 is a region set in a casewhere the floor of the periphery of the user 1 is provided with a stepor the like. In this case, using the height of the user 1 (physicalinformation) and the height of the floor position (environmentalinformation), a relative distance He of the lower display region 54 cwith respect to the reference horizontal plane 55 is set.

First of all, the environmental information acquisition unit 23 detectsthe position of the floor as the environmental information. For example,in a case where a step or the like lower than the position at which theuser 1 is standing is present, a height h₃ of the step is detected onthe basis of the depth data, the space map, or the like. Then, thelayout calculation unit 26 sets a relative distance H_(c) (H_(c)=h₁+h₃)of the lower display region 54 c at the same height as the lower floorfrom the height h₁ of the user 1 and the height h₃ of the step.Accordingly, even in a case where the floor has the step or the like,the image displayed along the floor like the virtual image 30 d can bedisplayed in a range easy for the user 1 to view. In this manner, inthis embodiment, the display position of the virtual image 30 d is setin accordance with the position of the floor.

For example, between a tall user 1 and a short user 1, the regions thatthe users 1 visually recognize are different even when the users 1 looksdown at the same angle of depression 7. Therefore, the region that theuser 1 can look down easily is different for each height of the user 1.Moreover, in a case of performing the display along the floor or thelike, the region in which the display can be easily visually recognizedchanges also depending on the difference of the height of the floor. Inthis embodiment, the display position of the lower image 36 iscontrolled in accordance with the height of the user 1 and the positionof the floor. Accordingly, irrespective of the height and the height ofthe floor, display easy for the user 1 to view can be realized, andexcellent usability can be provided.

It should be noted that the lower image 36 is, for example, displayed bythe user 1 performing the downward gesture of moving the head downward.The determination condition for determining this downward gesture (lowerthreshold angle β or the like) may be set in accordance with the heightof the user 1. For example, it is conceivable that when the user 1performs a motion of looking down the floor or the like, the motion (thepitch angle of the head) becomes larger as the height becomes higher. Inaccordance with such a feature, for example, adjustment to set the lowerthreshold angle β to be larger as the height becomes higher can beperformed. Accordingly, the lower image 36 or the like can be displayedby a natural motion irrespective of the height of the user 1.

Hereinabove, the processing using the field-of-view feature and themotion feature depending on the posture and the height of the user 1 hasbeen mainly described. Those features can differ in a manner thatdepends on the age and the gender of the user 1, for example. Forexample, the human field-of-view range and the range-of-motion differ ina manner that depends on the age. In view of such difference of thefeatures for each age, the setting of the determination condition, thecontrol of the display positions of the virtual images 30, and the likemay be performed. Moreover, those features have individual differences.For example, the tendence of the motion range or the like related to thegesture may be learned for each user 1 and the physical featureinformation of the individual user 1 may be calculated on the basis ofthe learning result. By performing the setting of the determinationcondition or the like in accordance with this physical featureinformation, the operability in virtual experience can be sufficientlyimproved.

Hereinafter, in the controller 20 according to this embodiment, thedetermination condition of the trigger gesture that is the trigger forthe operation input by the gesture is set on the basis of the physicalinformation of the user 1 wearing the HMD 100. Using this determinationcondition, whether or not the trigger gesture is performed is determinedon the basis of the gesture information and the output corresponding tothe trigger gesture is controlled on the basis of the determinationresult. Accordingly, the trigger gesture performed the user 1 isdetermined in accordance with the physical information of the user 1,and therefore the operability of gesture operations can be improved.

In this embodiment, the determination condition appropriate to the stateof the user 1 is set using the physical information such as the postureand height of the user 1. Moreover, the determination condition is setin accordance with the surrounding environment such as thepresence/absence of the ceiling. As described above, by setting thedetermination condition, the trigger gesture of the user 1 can bedetermined within a range in which the user 1 can naturally move or arange in which the user 1 can easily move. Accordingly, the user 1 caneasily perform the gesture operation irrespective of the state of theuser 1, and a user interface having a small physical burden can berealized.

Moreover, the display positions of the virtual images 30 are controlledin accordance with the physical information and the environmentalinformation. Accordingly, displaying the virtual image 30 in a mannereasy to view and displaying the virtual image 30 in a manner easy tofind can be performed, and high visibility can be ensured. Moreover, thedisplay easy for the user 1 to visually recognize can be performed, andhigh accessibility and usability can be realized.

Other Embodiments

The present technology is not limited to the above-mentionedembodiments, and various other embodiments can be realized.

Hereinabove, the trigger gestures for the display operation of causingthe virtual image to be displayed or the non-display operation ofcausing the virtual image not to be displayed have been described. Thepresent technology is not limited thereto, and a determination conditionfor determining a trigger gesture for other operation processing may beset on the basis of the physical information and the like of the user.

For example, the trigger gesture that is the trigger for the selectionoperation of selecting the virtual image may be used. In this case, thedetermination condition of the trigger gesture for the selectionoperation is set on the basis of the physical information of the user 1or the like. For example, in the virtual image 30 c shown in FIG. 13A, aplurality of windows is displayed and the user 1 selects a desiredwindow by the motion of moving the head leftward and rightward. Thethreshold angle with respect to the posture angle (yaw angle) thatchanges depending on this motion of moving the head leftward andrightward is set in accordance with the physical information of the user1 or the like. For example, in the standing state, the waist more easilyrotates as compared with the seated state, and therefore the yaw angleof the head can be moved widely. In accordance with such a feature, forexample, the threshold angle with respect to the yaw angle is set to belarger in the standing state as compared with the seated state.Accordingly, the selection operation can be performed by an easy motion,and erroneous detection and the like caused by a small threshold anglecan be avoided.

Moreover, the present technology is not limited to the gesture with thehead, and determination conditions about trigger gestures that the userperforms with the arms and legs may be set. For example, the virtualimage 30 c shown in FIG. 13A is displayed around the body of the user.The selection operation may be performed by reaching the user's hand tothe selection window 37 displayed as this virtual image 30 c. In thiscase, the user's gesture of reaching the hand beyond a predeterminedregion is the trigger gesture for the selection operation, and a rangeof the predetermined region is the determination condition(determination threshold). For example, in the standing state, a motionrange (motion feature of the hands) in which the user can move the handsis wider as compared with the seated state. Therefore, in the standingstate, a region that is the determination threshold is set to be widerthan in the seated state. Accordingly, the user can perform theselection operation in the range in which the user can easily move thehands irrespective of the posture.

Alternatively, the user may perform the selection operation by movingthe legs. For example, the virtual image 30 d shown in FIG. 13B isdisplayed near the legs of the user. The user may perform the selectionoperation by stepping on the operation window 39 included in thisvirtual image 30 d. In this case, the user's gesture of extending theleg beyond a predetermined region is the trigger gesture for theselection operation and a range of the predetermined region is thedetermination threshold. For example, in the standing state, a regionthat is the determination threshold is set to be wider than in theseated state. Accordingly, irrespective of whether the user is seated orstanding, the user can easily perform the selection operation by theleg. For example, such processing can be performed.

In addition, the types of the trigger gesture and the determinationthreshold are not limited. A determination threshold for determining thetrigger gesture may be set as appropriate, for example, in accordancewith the physical feature information of the user about a site (neck,hands, legs, waist, etc.) that moves when performing a trigger gesture.Moreover, in a case where the motion range or the like of each sitechanges due to a change of the environment, the determination thresholdis adjusted in accordance with the change. Accordingly, the gestureoperation with high operability can be realized in accordance with theuser's state and the surrounding situation.

Hereinabove, as the method of determining the trigger gesture, thethreshold processing using the determination threshold has been mainlydescribed. The present technology is not limited to the method ofdetermining the trigger gesture. For example, instead of the thresholdprocessing, the trigger gesture can also be determined by the use of aclassifier (learned model) built by machine learning such as deeplearning.

The classifier is capable of determining whether or not the triggergesture is performed by learning various parameters and the like basedon the machine learning algorithm, for example, by training using apredetermined data set. In this manner, the parameters that theclassifier learns can be considered as the determination condition fordetermining the trigger gesture. By inputting the gesture information ofthe user or the like into this classifier, whether or not the gesture isthe trigger gesture is determined and the gesture determination resultis output. The algorithm for building the classifier is not limited, andan arbitrary algorithm capable of detecting the type of the gesture, theamount of motion, and the like may be used.

For example, a data set in which gesture inputs (gesture information)are correlated to gesture determination results of the gesture inputs isused for training the classifier. In this case, using a plurality ofdata sets corresponding to the posture and the state such as the height,a plurality of classifiers appropriate to the respective states is builtand stored in the storage unit or the like. When determining the triggergesture, the gesture detection unit (control unit) selects and uses aclassifier appropriate to the user's state, i.e., the physical featureof the user on the basis of the physical information of the user. Forexample, a classifier for the seated state is used in a case where theuser is in the seated state, and a classifier for the standing state isused in a case where the user is in the standing state. Accordingly,determination processing of the trigger gesture that is appropriate tothe posture and the height of the user or the like can be performed, andthe operability of gesture operations can be improved.

Alternatively, for example, the classifier trained using the physicalfeature information such as the field-of-view range and the motion rangemay be used. In this case, the classifier has the gesture informationand the physical feature information as the input and outputs thegesture determination result. Accordingly, determination processing ofthe trigger gesture appropriate to the physical feature information ofthe user can be performed. Moreover, the physical feature of the userhave individual differences. The classifier may learn parametersdepending on the physical feature of each user on the basis of thephysical feature information of the individual user. Accordingly, theoperability of gesture operations can be greatly improved.

In the above-mentioned embodiment, virtual experience using augmentedreality (AR) is provided from the HMD (AR glasses) including thesee-through display. The present technology is not limited thereto, andfor example, the present technology can also be applied to an immersiveHMD or the like that realizes AR display by displaying an image of thereal space which has been taken through the rear-facing camera or thelike. Moreover, the present technology can also be applied in a casewhere virtual experience using virtual reality (VR) is provided by theuse of the immersive HMD or the like. In this case, the determinationcondition for determining the trigger gesture (determination thresholdor the like) is set by detecting the posture and the like of the userexperiencing VR.

Hereinabove, control of the image output corresponding to the triggergesture has been described. The output corresponding to the triggergesture is not limited to the image output, and the audio output or thetactile output corresponding to the trigger gesture may be controlled.For example, an operation of controlling the sound volume of sound orthe like output from the speaker mounted on the HMD by the gestureoperation may be capable of being performed. Alternatively, for example,in a case where a tactile presentation device such as a vibrationactuator is mounted, the type and strength of the tactile sense and thelike may be capable of being adjusted by the gesture operation. Thus,high operability can be provided by setting as appropriate thedetermination condition of the trigger gesture on the basis of thephysical information of the user even in a case where the audio outputor the tactile output is controlled.

As an apparatus using the head-mounted casing (head-mounted apparatus),the headphones, the earphones, or the like may be used. For example, bymounting the 9-axis sensor on the wireless headphones, the wiredheadphones, or the like, the motion of the head of the user and the likecan be detected. In this case, an operation such as sound volume controland music selection is performed as the gesture operation using thetrigger gesture as the trigger. Alternatively, in a case where thetactile presentation device is mounted on the headphones, the tactileoutput may be capable of being controlled by the gesture operation.

In addition, the present technology is not limited to the head-mountedtype apparatus, and the present technology can be applied to anarbitrary wearable device capable of detecting a gesture of the user.The user's gestures can be detected using, for example, devices to bemounted on the fingers, the wrist, the body, the waist, the ankles, orthe like. As described above, the amount of motion of the gesture of theuser and the like change depending on the physical feature of the userand the environment. For example, by changing the determinationcondition of the trigger gesture that each device is capable ofdetecting in accordance with the physical information and theenvironmental information, the trigger detection of the gestureoperation can be properly performed.

For instance, it is assumed that using devices mounted on the wrist andthe fingers, a hand gesture using the hands is detected. In this case,it is conceivable that the amount of motion of the hand gesture (e.g.,the range to move the arms) can change between the seated state and thestanding state. In each device, the determination condition of thetrigger gesture is set in accordance with such a change of the gesturethat depends on the posture. Accordingly, the gesture detection accuracycan be increased. In addition, the determination condition fordetermining the gesture that each device is capable of detecting may beset as appropriate.

Hereinabove, as the embodiment of the information processing apparatusaccording to the present technology, the head-mounted apparatus such asthe HMD and the headphones has been taken as the example. However, theinformation processing apparatus according to the present technology maybe realized by an arbitrary computer that is configured separate fromthe head-mounted apparatus and is connected to the head-mountedapparatus with a wire or wirelessly. For example, an external apparatusthat controls the head-mounted apparatus, such as a PC to be connectedto the HMD and a smartphone to be connected to a game console orheadphones may be used as the information processing apparatus accordingto the present technology. Alternatively, for example, the informationprocessing method according to the present technology may be performedby a cloud server. Alternatively, the information processing methodaccording to the present technology may be performed by cooperation ofthe HMD or the like with another computer.

That is, the information processing method and the program according tothe present technology may be performed not only in a computer systemconfigured by a single computer but also in a computer system in which aplurality of computers cooperatively operate. It should be noted that inthe present disclosure, the system means a set of a plurality ofcomponents (apparatus, module (parts), etc.) and it does not matterwhether or not all the components are housed in the same casing.Therefore, both of a plurality of apparatuses housed in separate casingsand connected to one another via a network and a single apparatus havinga plurality of modules housed in a single casing are the system.

Performing the information processing method and the program accordingto the present technology by the computer system includes, for example,both of a case where a single computer performs acquisition of thegesture information, setting of the determination condition of thetrigger gesture, and output control based on the determination result ofthe trigger gesture using the determination condition, and the like anda case where different computers execute these processes. Moreover,performing the respective processes by a predetermined computer includescausing another computer to execute some or all of those processes andacquire the results.

That is, the information processing method and the program according tothe present technology are also applicable to a cloud computingconfiguration in which a plurality of apparatuses shares andcooperatively processes a single function via a network.

At least two features of the features according to the presenttechnology which have been described above may be combined. That is, thevarious features described in the respective embodiments may bearbitrarily combined across the respective embodiments. Moreover, theabove-mentioned various effects are merely exemplary and not limitative,and other effects may be provided.

In the present disclosure, it is assumed that “the same”, “equal”,“orthogonal”, and the like are concepts including “substantially thesame”, “substantially equal”, “substantially orthogonal”, and the like.For example, also include states included in a predetermined range(e.g., ±10% range) using “completely the same”, “completely equal”,“completely orthogonal”, and the like as the basis.

It should be noted that the present technology can also take thefollowing configurations.

-   -   (1) An information processing apparatus, including:        -   a physical information acquisition unit that acquires            physical information of a user wearing a head-mounted            casing;        -   a gesture information acquisition unit that acquires gesture            information regarding a gesture of the user;        -   a determination condition setting unit that sets, on the            basis of the physical information, a determination condition            for determining a trigger gesture that is a trigger for an            operation input by the gesture of the user; and        -   a control unit that determines, on the basis of the gesture            information and the determination condition, whether or not            the trigger gesture is performed and controls, on the basis            of a result of the determination, an output corresponding to            the trigger gesture.    -   (2) The information processing apparatus according to (1), in        which        -   the determination condition setting unit sets the            determination condition on the basis of physical feature            information of the user, the physical feature information            corresponding to the physical information.    -   (3) The information processing apparatus according to (2), in        which        -   the physical feature information includes information            regarding at least one of a field-of-view feature or a            motion feature of the user.    -   (4) The information processing apparatus according to any one        of (1) to (3), in which        -   the determination condition includes a determination            threshold for determining the trigger gesture.    -   (5) The information processing apparatus according to any one        of (1) to (4), in which        -   the physical information acquisition unit acquires a posture            of the user as the physical information, and        -   the determination condition setting unit sets the            determination condition in accordance with the posture of            the user.    -   (6) The information processing apparatus according to (5), in        which        -   the determination condition setting unit sets the            determination condition in accordance with at least one of a            field-of-view feature or a motion feature in the posture of            the user.    -   (7) The information processing apparatus according to (5) or        (6), in which        -   the trigger gesture is a head gesture of the user moving the            head upward,        -   the determination condition includes a threshold angle with            respect to a posture angle of the head of the user, the            posture angle changing in a manner that depends on the head            gesture,        -   the physical information acquisition unit detects a standing            state or a seated state as the posture of the user, and        -   the determination condition setting unit sets, in a case            where the seated state is detected, the threshold angle to            be a smaller value than in a case where the standing state            is detected.    -   (8) The information processing apparatus according to any one        of (1) to (7), in which        -   the head-mounted casing is a casing for a head-mounted            display that displays a virtual image, and        -   the control unit controls a display position of the virtual            image on the basis of the physical information.    -   (9) The information processing apparatus according to (8), in        which        -   the physical information acquisition unit acquires a posture            of the user as the physical information,        -   the control unit controls the display position of the            virtual image in accordance with the posture of the user.    -   (10) The information processing apparatus according to (9), in        which        -   the virtual image is an upper image to be displayed on a            diagonally upper side as viewed from the user,        -   the physical information acquisition unit detects a standing            state or a seated state as the posture of the user, and        -   the control unit sets, in a case where the seated state is            detected, the display position of the upper image to be a            lower position than in a case where the standing state is            detected.    -   (11) The information processing apparatus according to any one        of (8) to (10), in which        -   the physical information acquisition unit acquires a height            of the user as the physical information,        -   the virtual image is a lower image to be displayed on a            diagonally lower side as viewed from the user,        -   the control unit sets a display position of the lower image            in accordance with the height of the user.    -   (12) The information processing apparatus according to any one        of (1) to (11), further including        -   an environmental information acquisition unit that acquires            environmental information regarding a surrounding            environment of the user.    -   (13) The information processing apparatus according to (12), in        which        -   the determination condition setting unit sets the            determination condition on the basis of the environmental            information.    -   (14) The information processing apparatus according to (13), in        which        -   the trigger gesture is a head gesture of the user moving the            head upward,        -   the determination condition includes a threshold angle with            respect to a posture angle of the head of the user, the            posture angle changing in a manner that depends on the head            gesture,        -   the environmental information acquisition unit detects            presence/absence of a ceiling in the surrounding            environment, and        -   the determination condition setting unit sets, in a case            where the ceiling is detected, the threshold angle to be a            smaller value than in a case where the ceiling is not            detected.    -   (15) The information processing apparatus according to any one        of (12) to (14), in which        -   the head-mounted casing is a casing for a head-mounted            display that displays a virtual image, and        -   the control unit controls a display position of the virtual            image on the basis of the environmental information.    -   (16) The information processing apparatus according to (15), in        which        -   the environmental information acquisition unit determines            whether or not the surrounding environment includes a            vanishing point, and        -   the control unit sets, in a case where the surrounding            environment includes the vanishing point, the display            position of the virtual image by using the vanishing point            as a basis.    -   (17) The information processing apparatus according to (15) or        (16), in which        -   the environmental information acquisition unit detects a            position of a floor as the environmental information, and        -   the control unit sets the display position of the virtual            image in accordance with the position of the floor.    -   (18) The information processing apparatus according to any one        of (1) to (17), in which        -   the head-mounted casing is a casing for a head-mounted            display that displays a virtual image, and        -   the trigger gesture functions as a trigger for a display            operation of causing the virtual image to be displayed, a            trigger for a non-display operation of causing the virtual            image not to be displayed, or a trigger for a selection            operation of selecting the virtual image.    -   (19) An information processing method, including:        -   by computer system,        -   acquiring physical information of a user wearing a            head-mounted casing;        -   acquiring gesture information regarding a gesture of the            user;        -   setting, on the basis of the physical information, a            determination condition for determining a trigger gesture            that is a trigger for an operation input by the gesture of            the user; and        -   determining, on the basis of the gesture information and the            determination condition, whether or not the trigger gesture            is performed and controlling output corresponding to the            trigger gesture on the basis of a result of the            determination.    -   (20) A program that causes a computer system to execute:        -   a step of acquiring physical information of a user wearing a            head-mounted casing;        -   a step of acquiring gesture information regarding a gesture            of the user;        -   a step of setting, on the basis of the physical information,            a determination condition for determining a trigger gesture            that is a trigger for an operation input by the gesture of            the user; and        -   a step of determining, on the basis of the gesture            information and the determination condition, whether or not            the trigger gesture is performed and controlling output            corresponding to the trigger gesture on the basis of a            result of the determination.

REFERENCE SIGNS LIST

-   -   1 user    -   5 head-mounted casing    -   10 display unit    -   14 sensor unit    -   15 storage unit    -   20 controller    -   21 physical information acquisition unit    -   22 gesture information acquisition unit    -   23 environmental information acquisition unit    -   24 determination condition setting unit    -   25 gesture detection unit    -   26 layout calculation unit    -   27 layout determination unit    -   28 output control unit    -   30, 30 a to 30 g virtual image    -   31 upper image    -   36 lower image    -   51 vanishing point    -   100 HMD

The invention claimed is:
 1. An information processing apparatus,comprising: circuitry configured to: detect, as a posture of a user whowears a head-mounted casing, one of a standing state of the user or aseated state of the user; acquire gesture information regarding agesture of the user; set, based on the posture of the user, adetermination condition to determine a trigger gesture that is a triggerfor an operation input by the gesture of the user, wherein the triggergesture includes a head gesture of moving a head of the user upward, thedetermination condition includes a threshold angle with respect to aposture angle of the head of the user, and the posture angle ischangeable based on the head gesture; set, in a case where the seatedstate is detected, the threshold angle to be a smaller value than in acase where the standing state is detected; determine, based on thegesture information and the determination condition, whether the triggergesture is performed; and control, based on a result of thedetermination, an output corresponding to the trigger gesture.
 2. Theinformation processing apparatus according to claim 1, wherein thecircuitry is further configured to set the determination condition basedon physical feature information of the user.
 3. The informationprocessing apparatus according to claim 2, wherein the physical featureinformation includes information regarding at least one of afield-of-view feature or a motion feature of the user.
 4. Theinformation processing apparatus according to claim 1, wherein thedetermination condition further includes a determination threshold todetermine the trigger gesture.
 5. The information processing apparatusaccording to claim 1, wherein the circuitry is further configured to setthe determination condition based on at least one of a field-of-viewfeature or a motion feature in the posture of the user.
 6. Theinformation processing apparatus according to claim 1, wherein thehead-mounted casing is a casing for a head-mounted display that displaysa virtual image.
 7. The information processing apparatus according toclaim 6, wherein the circuitry is further configured to control adisplay position of the virtual image based on the posture of the user.8. The information processing apparatus according to claim 7, whereinthe virtual image is an upper image to be displayed on a diagonallyupper side as viewed from the user, and the circuitry is furtherconfigured to set, in the case where the seated state is detected, thedisplay position of the upper image to be a lower position than in thecase where the standing state is detected.
 9. The information processingapparatus according to claim 6, wherein the circuitry is furtherconfigured to: acquire physical information indicating a height of theuser, wherein the virtual image is a lower image to be displayed on adiagonally lower side as viewed from the user, and set a displayposition of the lower image based on the height of the user.
 10. Theinformation processing apparatus according to claim 1, wherein thecircuitry is further configured to acquire environmental informationregarding a surrounding environment of the user.
 11. The informationprocessing apparatus according to claim 10, wherein the circuitry isfurther configured to set the determination condition based on theenvironmental information.
 12. The information processing apparatusaccording to claim 11, wherein the circuitry is further configured to:detect one of a presence or an absence of a ceiling in the surroundingenvironment, and set, in a case where the ceiling is present in thesurrounding environment, the threshold angle to be the smaller valuethan in a case where the ceiling is absent in the surroundingenvironment.
 13. The information processing apparatus according to claim10, wherein the head-mounted casing is a casing for a head-mounteddisplay that displays a virtual image, and the circuitry is furtherconfigured to control unit a display position of the virtual image basedon the environmental information.
 14. The information processingapparatus according to claim 13, wherein the circuitry is furtherconfigured to: determine the surrounding environment includes avanishing point, and set, based on the determination that thesurrounding environment includes the vanishing point, the displayposition of the virtual image by using the vanishing point.
 15. Theinformation processing apparatus according to claim 13, wherein thecircuitry is further configured to: detect a position of a floor as theenvironmental information, and set the display position of the virtualimage based on the position of the floor.
 16. The information processingapparatus according to claim 1, wherein the head-mounted casing is acasing for a head-mounted display that displays a virtual image, and thetrigger gesture functions as one of a trigger for a display operation ofcausing the virtual image to be displayed, a trigger for a non-displayoperation of causing the virtual image not to be displayed, or a triggerfor a selection operation of selecting the virtual image.
 17. Aninformation processing method, comprising: by a computer system,detecting, as a posture of a user wearing a head-mounted casing, one ofa standing state of the user or a seated state of the user; acquiringgesture information regarding a gesture of the user; setting, based onthe posture of the user, a determination condition for determining atrigger gesture that is a trigger for an operation input by the gestureof the user, wherein the trigger gesture includes a head gesture ofmoving a head of the user upward, the determination condition includes athreshold angle with respect to a posture angle of the head of the user,and the posture angle is changeable based on the head gesture; setting,in a case where the seated state is detected, the threshold angle to bea smaller value than in a case where the standing state is detected;determining, based on the gesture information and the determinationcondition, whether the trigger gesture is performed; and controlling anoutput corresponding to the trigger gesture based on a result of thedetermination.
 18. A non-transitory computer-readable medium havingstored thereon, computer-executable instructions which, when executed bya computer, cause the computer to execute operations, the operationscomprising: detecting, as a posture of a user wearing a head-mountedcasing, one of a standing state of the user or a seated state of theuser; acquiring gesture information regarding a gesture of the user;setting, based on the posture of the user, a determination condition fordetermining a trigger gesture that is a trigger for an operation inputby the gesture of the user, wherein the trigger gesture includes a headgesture of moving a head of the user upward, the determination conditionincludes a threshold angle with respect to a posture angle of the headof the user, and the posture angle is changeable based on the headgesture; setting, in a case where the seated state is detected, thethreshold angle to be a smaller value than in a case where the standingstate is detected; determining, based on the gesture information and thedetermination condition, whether the trigger gesture is performed; andcontrolling an output corresponding to the trigger gesture based on aresult of the determination.
 19. An information processing apparatus,comprising: circuitry configured to: acquire physical information of auser who wears a head-mounted casing, wherein the physical informationindicates a height of the user, the head-mounted casing is a casing fora head-mounted display that displays a virtual image, and the virtualimage is a lower image to be displayed on a diagonally lower side asviewed from the user; set a display position of the lower image based onthe height of the user; acquire gesture information regarding a gestureof the user; set, based on the physical information, a determinationcondition to determine a trigger gesture that is a trigger for anoperation input by the gesture of the user; determine, based on thegesture information and the determination condition, whether the triggergesture is performed; and control, based on a result of thedetermination, an output corresponding to the trigger gesture.